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Preface 


People have gambled and played games for thousands of years. 
Yet, only in the 17th century we see a serious attempt for a 
scientific approach to the subject. The combinatorial foundations of 
probability theory were developed by various mathematicians such 
as J. BERNOULLI! [4] as a means to understand games of chance 
(mostly involving rolls of dice) and to make conjectures according to 
mathematical principles. 

Since then, game theory has grown into a wide field and appears 
at times quite removed from its combinatorial roots. The notion of a 
game has been broadened to encompass all kinds of human behavior 
and interactions of individuals or of groups and societies (see, e.g., 
BERNE [3]). Much of current research studies humans in economic 
and social contexts and seeks to discover behavioral laws in analogy 
to physical laws. 

The role of mathematics in this endeavor, however, has been quite 
limited so far. One major reason lies certainly in the fact that players 
in real life often behave differently than a simple mathematical 
model would predict. So seemingly paradoxical situations exist where 
people appear to contradict the straightforward analysis of the 
mathematical model builder. A famous such example is the chain 
store paradox of SELTEN? [41]. 


J. BERNOULLI (1654-1705) 
?R. SELTEN (1930-2016) 
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This is not withstanding the ground breaking work of VON NEv- 
MANN and MORGENSTERN? [34], who have proposed an axiomatic 
approach to notions of utilities and rational behavior of the players 
of a game. 

As interesting and worthwhile as research into laws that govern 
psychological, social or economic behavior of humans may be, the 
present Mathematical Game Theory is not about these aspects of 
game theory. In the center of our attention are mathematical models 
that may be useful for the analysis of game-theoretic situations. 
We are concerned with the mathematics of game-theoretic models 
but leave the question aside whether a particular model describes a 
particular situation in real life appropriately. 

The mathematical analysis of a game-theoretic model treats 
objects neutrally. Elements and sets have no feelings per se and 
show no psychological behavior. They are neither generous nor cost 
conscious unless such features are built into the model as clearly 
formulated mathematical properties. The advantage of mathematical 
neutrality is substantial, however, because it allows us to embed the 
mathematical analysis into a much wider framework. 

This introduction into mathematical game theory sees games 
being played on (possibly quite general) systems. A move of a game 
then correspond to a transition of a system from one state to another. 
Such an approach reveals a close connection with fundamental 
physical systems via the same underlying mathematics. Indeed, it 
is hoped that mathematical game theory may eventually play a role 
for real world games akin to the role of theoretical physics to real 
world physical systems. 

The reader of this introductory text is expected to have knowl- 
edge in mathematics, perhaps at the level of a first course in linear 
algebra and real analysis. Nevertheless, the text will review relevant 
mathematical notions and properties and point to the literature for 
further details. 


3J. von NEUMANN (1903-1953), O. MORGENSTERN (1902-1977) 


Preface xi 


The reader is furthermore expected to read the text “actively”. 
“Ex.” marks not only an “example” but also an “exercise” that might 
deepen the understanding of the mathematical development. 

The book is based on a one-term course on the subject the author 
has presented repeatedly at the University of Cologne to pre-master 
and master level students with an interest in applied mathematics, 
operations research and mathematical modelling. 

It is dedicated to the memory of WALTER KERN.* 


4W. KERN (1957-2021) 
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Chapter 1 


Mathematical Models of the Real World 


This introductory chapter discusses mathematical models, sketches the 
mathematical tools for their analysis, defines systems in general and 


systems of decisions in particular. Games are introduced from a general 
point of view and it is indicated how they may arise in combinatorial, 
economic, social, physical and other contexts. 


1. Mathematical modelling 


Mathematics is the powerful human instrument to analyze and to 
structure observations and to possibly discover natural “laws”. These 
laws are logical principles that allow us not only to understand 
observed phenomena (i.e., the so-called real world) but also to 
compute possible evolutions of current situations, and thus to 
attempt a “look into the future”. 

Why is that so? An answer to this question is difficult if not 
impossible. There is a wide-spread belief that mathematics is the 
language of the universe.‘ So everything can supposedly be captured 
by mathematics and all mathematical deductions reveal facts about 
the real world. I do not know whether this is true. Even if it were, one 
would have to be careful with real-world interpretations of mathe- 
matics, nonetheless. A simple example may illustrate the difficulty: 

While apples on a tree are counted in terms of natural numbers, 
it would certainly be erroneous to conclude: for every natural number 


‘GALILEO GALILEI (1564-1642). 
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n, there exists a tree with n apples. In other words, when we use the 
set of nonnegative integers to describe the number of apples on a 
tree, our mathematical model will comprise mathematical objects 
that have no real counterparts. 

Theoretically, one could try to get out of the apple dilemma by 
restricting the mathematical model to those numbers n that are 
realized by apple trees. But such a restricted model would be of 
no practical use as neither the set of such apple numbers n nor 
its specific algebraic structure is explicitly known. Indeed, while the 
sum m+n of two natural numbers m and n is a natural number, 
it is not clear whether the existence of two apple trees with m 
resp. n apples guarantees the existence of an apple tree with m+n 
apples. 

In general, a mathematical model of a real-world situation is, alas, 
not necessarily guaranteed to be absolutely comprehensive. Math- 
ematical conclusions are possibly only theoretical and may suggest 
objects and situations which do not exist in reality. One always has to 
double-check real-world interpretations of mathematical deductions 
and ask whether an interpretation is “reasonable” in the sense that 
it is commensurate with one’s own personal experience. 

In the analysis of a game-theoretic situation, for example, one 
may want to take the psychology of individual players into account. 
A mathematical model of psychological behavior, however, is typi- 
cally based on assumptions whose accuracy is unclear. Consequently, 
mathematically established results within such models must be 
interpreted with care, of course. 

Moreover, similar to physical systems with a large number 
of particles (like molecules), game-theoretic systems with many 
agents (e.g., traffic systems and economies) are too complex to 
analyze by following each of the many agents individually. Hence 
a practical approach will have to concentrate on “group behavior” 
and consider statistical parameters that average over individual 
numerical attributes. 

Having cautioned the reader about the real-world interpretation 
of mathematical deductions, we will concentrate on mathematical 
models (and their mathematics) and leave the interpretation to the 
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reader. Our emphasis is on game-theoretic models. So we should 
explain what we understand by this. 

A game involves players that perform actions which make a given 
system go through a sequence of states. When the game ends, the 
system is in a state according to which the players receive rewards 
(or are charged with costs or whatever). Many game theorists think 
of a “player” as a humanoid, 7.e., a creature with human feelings, 
wishes and desires, and thus give it a human name.” 

Elements of a mathematical model, however, do not have 
humanoid feelings per se. If they are to represent objects with wishes 
and desires, these wishes and desires must be explicitly formulated 
as mathematical optimization challenges with specified objective 
functions and restrictions. Therefore, we will try to be neutral and 
refer to “players” often just as agents with no specified sexual 
attributes. In particular, an agent will typically be an “it” rather 
than a “he” or a “she”. 

This terminological neutrality makes it clear that mathematical 
game theory comprises many more models than just those with 
human players. As we will see, many models of games, decisions, 
economics and social sciences have the same underlying mathematics 
as models of physics and informatics. 


Note on continuous and differentiable functions. Real world 
phenomena are often modelled with continuous or even differentiable 
functions. However, 


e There exists no practically feasible test for the continuity 


or the differentiability of a function! 


Continuity and differentiability, therefore, are assumptions of the 
model builder. These assumptions appear often reasonable and 
produce good results in applications. Moreover, they facilitate the 
mathematical analysis. Yet, their appropriateness cannot be proven 


? Alice and Bob are quite popular choices. 
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by tests and experiments. The reader should be aware of the 
difference between a mathematical model and its physical origin. 


Note on algorithms and computational complexity. Game- 
theoretic questions naturally call for mathematical computations 
within appropriate models. This will become clear in the present 
text, which also tries to exhibit important links to mathematical 
optimization theory. However, here is not the place to discuss specific 
mathematical optimization procedures per se. There is an abundance 
of classical mathematical literature on the latter, which can be 
consulted by the interested reader. 

The question of the complexity of computations within particular 
game-theoretic models has attracted the interest of theoretical 
computer science and created the field of algorithmic game theory,? 
whose details would exceed the frame and aim of this text and are, 
therefore, not addressed either. 


2. Mathematical preliminaries 


The reader is assumed to have basic mathematical knowledge (at 
least at the level of an introductory course on linear algebra). Never- 
theless, it is useful to review some of the mathematical terminology. 
Further basic facts are outlined in the Appendix. 


2.1. Functions and data representation 


A function f : S — W assigns elements f(s) of a set W as values 
to the elements s of a set S. One way of looking at a function is to 
imagine a measuring device “f” which produces the result f(s) upon 


the input s: 
seS — — fi(s)ew. 


We denote the collection of all W-valued functions with domain S' as 


We ={f:S>W} 


3See, e.g., NISAN et al. [35]. 
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and think of an element f € W® also as a parameter vector whose 
coordinates f, are indexed by the elements s € S and have values 
fs = f(s) € W. 

There is a dual way of looking at this situation where the roles 
of the function f and the variable s are reversed. The dual viewpoint 
sees s as a probe which produces the value f(s) when exposed to f: 


f—€ Ea. f(s). 


If S is small, the function f can be presented by a table which displays 
the total effect of f on S: 


peo (eb 
Fo |Fea Foster) 


The dual viewpoint would fix an element s € S and evaluate the effect 
of the measuring devices f1,..., fx, for example, and thus represent 
an individual element s € S by a k-dimensional data table: 


| fi | fe | fs |e] fe | 
fils)|fo(s)|f3(s)|---[fe(s) 


The dual viewpoint is typically present when one tries to describe 
the state o of an economic, social or physical system G via the data 
values fi(7), fo(o), ..., fe(o) of statistical measurements f1,..., fx 
with respect to k system characteristics: 


OC Se (filo), fa(o),---, f(a). 


Ss 


The two viewpoints are logically equivalent. Indeed, the dual per- 
spective sees the element s € S just like a function §: W° ~ W 
with values 


Also the first point of view is relevant for data representation. 
Consider, for example, a n-element set 


NE aca 
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In this context, a function f : N > {0,1} may represent a subset S 
of N, via the identification 


Fe{OyY + Sp=fieN| FP=YCN. 


REMARK 1.1. The vector f in (1) is the incidence vector of the 
subset Sy. Denoting by N the collection of all subsets of N and 
writing 2 = {0,1}, the context (1) establishes the correspondence 


oV={0,1})" 4s N={SC NI}. 


Nota BENE. (0, 1)-valued functions can also have other inter- 
pretations, of course. Information theory, for example, thinks 
of them as bit vectors and thus as carriers of information. 


An abstract function is a purely mathematical object with no 
physical meaning by itself. It obtains a concrete meaning only 
within the modelling context to which it refers. 


Ex. 1.1. The so-called HEISENBERG and the SCHRODINGER pic- 


tures of quantum theory are dual to each other in the sense of the 


present section.* 


Notation. When we think of a function f : S > W as a data 
representative, we think of f as a coordinate vector f € W* with 
coordinate components f, = f(s) and also use the notation 


f =(fsls € S). 
In the case S = {5 1, 52,53...}, we may write 


f= (Fba) Ff Bale as) = (Tisleoseee) = (fsls € S) 


Matrices. The direct product of two sets X and Y is the set of all 
ordered pairs (x,y) of elements « € X andy €Y, i.e., 


XxY={(z,y)|cex,yeY}. 


4 cf. Chapter 9.4.1. 
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A function A: X x Y ~ W can be imagined as a matrix with rows 
labeled by the elements x € X and columns labeled by the elements 
yeY: 


Agoy Azoys Azsys ty 
Aggy, Azzy Azsys a | 


Agiy, Agi y Agyys * | 


A= 


The function values Az, are the coefficients of A. We express this 
point of view in the shorthand notation 


A=|A; ee: 


The matrix form suggests to relate similar structures to A. The 
transpose of the matrix A € W**’, for example, is the matrix 


AT = [Ai] © W*** with the coefficients AJ, = Agy. 


In the case X = {1,2,...,m}and Y = {1,2,...,n}, one often simply 
writes 


wmrxn ~ Ww*xy 


REMARK 1.2 (Row and column vectors). When one thinks of 
a coordinate vector f € W* as a matrix having just f as the only 
column, one calls f a column vector. If f corresponds to a matrix 
with f as the only row, f is a row vector. So: 


f? row vector <= f column vector. 


Graphs. A (combinatorial) graph G = G(X) consists of a set X of 
nodes (or vertices) whose ordered pairs (x,y) of elements are viewed 
as arrows or (directed) edges between nodes: 


Q-®% (x,y EX). 


Denoting, as usual, the set of all real numbers by R, an R-weighting 
of G is an assignment of real number values a, to the nodes x € X 


10 1. Mathematical Models of the Real World 


and az, to the other edges (x,y) and hence corresponds to a matrix 
y 
Aen 


with X as its row and its column index set and coefficients Azy = ayy. 


REMARK 1.3. Although logically equivalent to a matrix, a graph 
representation of data is often more intuitive in dynamic contexts. 

A directed edge e = (x,y) may, for example, represent a road 
along which one can travel from «x to y in a traffic context. In another 
context, e could also indicate a possible transformation of x into y, 
etc. 

The edge weight a, = az, could be the distance from x to y or 
the strength of an action exerted by x onto y, etc. 


2.2. Algebra of functions and matrices 


While the coefficients of data vectors or matrices can be quite varied 
(colors, sounds, configurations in games, etc.), we will typically deal 
with numerical data so that coordinate vectors have real numbers as 
their component values. Hence we deal with coordinate spaces of the 


type 


R® = {f: SR}. 


Addition and scalar multiplication. The sum f + g of two 
coordinate vectors f,g € R®° is the vector of component sums 


(f+9)s = fst Qs, 1.€., 
f+g=(fst+gs|s € S). 


For any scalar A € R, the scalar product Af multiplies each 
component of f € R® by 2: 


Af = Afels € 8). 


WARNING. There are many — quite different — notions for “multi- 
plication” operations with vectors. 
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Products. The (function) product f eg of two vectors f,g € R® is 
the vector with the componentwise products, 7.e., 


feg=(fsgsls € S). 


In the special case of matrices A,B € R*** the function product of 
A and B is called the HADAMARD® product 


AeBeR**” (with coefficients (Ae B) ny = Ary Bay). 


WARNING. The HADAMARD product is quite different than the 
standard matriz multiplication rule (3) below. 


Norm and inner products. Data vectors have typically only 
finitely many components. This means: One deals with parameter 
vectors f € R° where S is a finite set. Assuming S to be finite, the 
following notions are mathematically well-defined since they involve 
sums with only finitely many summands: 

The (euclidian) norm of a vector f € R® is the parameter 


Fl =, fo fs. 
ses 


The inner product of the vectors f,g € R® is the real number (!): 


(flg} = S> fsgs = SOF °9)s, 


ses ses 


which allows us to express the norm as 


fll = V FIP). 


For finite matrices A, B € R***, we have the completely analogous 
notions: 


(AIB) = 37 7 AnyBoy and ||Al/? = (AA). 
xEX yEY 


°J. HADAMARD (1865-1963). 
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Orthogonality. The idea of a norm is motivated by geometric 
considerations which suggest to interpret the norm ||f|| as the length 
of the vector f. One says that f,g € R® are orthogonal if they satisfy 
the so-called Theorem of PYTHAGORAS, 7.e., the relation 


FIP + llgll? = If + gll?. (2) 


REMARK 1.4. Notice that the “Theorem of PyTHAGORAS” (2) 
is actually not a theorem but a definition (for the notion of 
orthogonality) in the context of the algebraic structure R*. 


LEMMA 1.1 (Orthogonality). Assuming S' finite, one has 
for the coordinate vectors f,g € R°: 


f and g are orthogonal <> (flg) =0. 


Proof. Straightforward exercise. 


The standard matrix product. If we have matrices A € R*** 
and B € R**4, every row vector A, of A is an element of R* 
and every column vector B* of B is an element of R*. So the 
corresponding inner products 


(Ar|B?) = S> Ary Byz 
yeY 


are well-defined. The matrix AB € R**% of all those inner products 
is called the (standard) product of A and B. 


The standard product of matrices A € R*** and B € RU*4 
is ONLY declared in the case U = Y and, then, defined as the 
matrix 


C=ABER**% with coefficients C,, = y lapis 18) 
yeY 
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If we think of f,g € R® as column vectors, then f” defines a matrix 
with just one row f7 and g a matrix with just one column. With this 
understanding, we find 


f'9=S— feos = (f\g)- 


ses 


Ex. 1.2. If f,g € R” are column vectors, then f’q is formally a 
(1 x 1)-matrix with the single coefficient (f|g). This is to be clearly 
distinguished from the (n x n)-matrix 


| fio fig2 ---  fign 
fog. = fog2 «es fn 
eeu E R°*”, 


RENEE fng2 see fnIn 
Ex. 1.3 (Trace). The trace tr C of a matrix C' is the sum of its 
diagonal elements. Show for the matrices A,B € R**”: 
(A|B) = tr (BTA) = S7(BTA)yy. 
yeY 


fg 


Rmxr 


Linear maps. Recall that one may think of a matrix A € as 


the parametrization of a linear map f : R” > R” that assigns to the 
n unit vectors u; € R” the n column vectors of A: 


Ava 
A general member of R” is a linear combination of the n unit vectors 
u; with scalar coefficients A;. Linearity of f means: 


fst +++ + Amtn) = Arf (ur) +++ + Ant (un) -y Au 


and hence 


f(x) = Az for all (column vectors) « € R™. 
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REMARK 1.5. Where J € R"*” is the (n x n) identity matrix (with 
column vectors u;), one has 


AI=A. 


In contrast, assuming A € R”*", the HADAMARD product yields a 
diagonal matrix: 


Aut 0 ee | 


ete 


Hilbert spaces. If S' is an infinite set, both the sum f + g and the 
HADAMARD product f eg are well-defined for any f,g € R®°. However, 
the notions of norms and inner products become vague because of 
the infinite sums in their definitions. 


As a way around this difficulty, let us call a parameter vector f € R® 


a HILBERT® vector if 


(H;) Only a countable number of coefficients f; of f are 
non-zero. 


(Hz) |IFI? = >_ fel? < 00. 


ses 


Let £2(S') denote the so-called HILBERT space of all HILBERT vectors 
in R®. (S$) is a vector space over the scalar field R in the sense 


(V) f,g€&(S) andACR = Aft+gE&(S). 


As one knows form the algebra of series with at most countably 
many summands, norms and inner products are well-defined scalar 
numbers for all vectors in the HILBERT space (2(S). 


°D. HILBERT (1862-1943). 
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While much of mathematical game theory can be developed in the 
more general setting of HILBERT spaces, we will be mostly concerned 
with finite-dimensional coordinate spaces in this introductory text. 


Ex. 1.4. Show that all finite-dimensional real coordinate spaces are 
HILBERT spaces. In fact, one has: 


é(S)=R° <> |S|<oo. 


2.3. Numbers and algebra 


The set R of real numbers has an algebraic structure under the usual 
addition and multiplication rules for real numbers. R contains the 
set of natural numbers 


i\ 2 pi eee earee, 


The computational rules of R may also be applied to N because sums 
and products of two natural numbers yield natural numbers.’ Similar 
algebraic rules can be defined on other sets. We give two examples 
below. 


REMARK 1.6. There is the philosophical issue whether “0” is a 
natural number, which corresponds to the question whether an entity 
can be a “set” when it is empty, i.e., contains no element.® For 
clarification, we therefore employ the notation 


No = NU {0} 
for the set of natural numbers including 0. 
Complex numbers. There is no real number r € R with the 
property r? = —1. To remedy this deficiency, one may introduce 


a new “number” i and do computations with it like it were a real 
number with the property 


?=-1. 
“Though the same is not guaranteed for subtractions and divisions, of course. 


8In Europe, for example, the idea and mathematical use of a number “zero” 
became customary not before the 13th century. 
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In doing so, one arrives at more general numbers of the form z = 
a+ib, with a and 6b being real numbers. The set 


C={a+ib|a,beR} 
is the set of complex numbers. The special number 
i=0+i-1 


is the so-called imaginary unit. Using the algebraic rules of R and 
always keeping i2 = —1 in mind, one can perform the usual com- 
putations with additions, subtractions, multiplications and divisions 
in C. 


Ex. 1.5. Assume that numbers a, b,c,d € R are given such that 


1 
. gh : ey as ee 
(a+ ib)(c+id) = 1, i.e, c+id = (a+ib)~ = aay 


Express the real numbers c and d in terms of the real numbers 
a and 0b. 


A sober, neutral look at the matter reveals that we have imposed an 


algebraic structure on the set R x R of pairs (a,b) of real numbers 
with the computational rules 


(a,b) + (c,d) = (a+c,b4+d) 


(a,b) - (c,d) = (ac — bd, ad + bc). 


If we identify the pair (1,0) with the real number 1 and the pair 
(0,1) with the imaginary unit i, we observe 


(a,b) = a(1,0) + 0(0,1)ERxR <3 a+ibeEC. 


It is straightforward to check that the rules (4) correspond precisely 
to the computational rules for complex numbers. 
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Another interesting view on complex numbers is offered by their 
representation as (2 x 2)-matrices: 


bl © 
0 1 


a+ibeC +> a| 


Ex. 1.6. Show: The sum and the product of two complex numbers 
is compatible with the matrix sum and (standard) matrix product 
of their representation of type (5). 


We will see in Chapter 9 how the matrix representation (5) captures 
the basic idea of interaction among two agents and provides the 
fundamental link to the mathematical model underlying quantum 
theory. 

For computational purposes, we retain: 


Algebra in C follows the same rules as algebra in R with the 


additional rule 


ile 


Binary algebra. Define an addition 6 and a multiplication ® on 
the 2-element set B = {0,1} according to the following tables: 


Bio 1 @|0 1 
0/0 1 and O/0 O 
1/1 0 1/0 1 


Also in this binary algebra, division is possible in the sense that the 
equation 


r®Oy=l 


has a unique solution y “for every” x 4 0.9 


°There is actually only one such case: y = a = 1. 
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Vector algebra. Complex numbers allow us to define sums and 
products of vectors with complex coefficients in analogy with real 
sums and products. 

The same is true for vectors with (0,1)-coefficients under the 
binary algebraic rules.!° 


REMARK. Are there clearly defined “correct” or “optimal” 
addition and multiplication rules on data structures that 
would reveal their real-world structure mathematically? 


The answer is “no” in general. The imposed algebra is always 
a choice of the mathematical analyst — and not of “mother 
nature”. It often requires care and ingenuity. Moreover, dif- 


ferent algebraic setups may reveal different structural aspects 
and thus lead to additional insight. 


2.4. Probabilities, information and entropy 


Consider n mutually exclusive events £1,...,,, and expect that any 
one of these, say F;, indeed occurs “with probability” p; = Pr(£;). 
Then the parameters p; form a probability distribution p € R€ on the 
set € = {F\,..., En}, i.e., the p; are nonnegative real numbers that 
sum up to 1: 


ppte:++pPn=1 and py,...,Pn > 0. 


If we have furthermore a measuring or observation device f that 
produces the number f; if FE; occurs, then these numbers have the 
expected value 


u(f) = fipi +--+ + frPn = > fii = (fp). (6) 


k=1 


In a game-theoretic context, a probability is often a subjective 
evaluation of the likelihood for an event to occur. The gambler, 


1 An application of binary algebra is the analysis of winning strategies for nim 
games in Section 2.6. 
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investor, or general player may not know in advance what the future 
will bring, but has more or less educated guesses on the likelihood 
of certain events. There is a close connection with the notion of 
information. 


Intensity. We think of the intensity of an event FE as a numerical 
parameter that is inversely proportional to its probability p = Pr(F) 
with which we expect its occurrence to be: the smaller p, the more 
intensely felt is an actual occurrence of &. For simplicity, let us take 
1/p as our objective intensity measure. 


REMARK 1.7 (FECHNER’S law). According to FECHNER,!! the 
intensity of a physical stimulation is physiologically felt on a 
logarithmic scale. Well-known examples are the Richter scale for 
earthquakes or the decibel scale for the sound. 


Following FECHNER, we feel the intensity of an event EF that we 
expect with probability p on a logarithmic scale and hence according 
to a function of type 


Ta(p) = log, (1/p) = log, D; (7) 


where log, p is the logarithm of p relative to the basis a > 0 (see 
Ex. 1.7). In particular, the occurrence of an “impossible” event, which 
we expect with zero probability, has infinite intensity 


I,(0) = — log, 0 = +00. 


Nota BENE. The mathematical intensity of an event depends 
only on the probability p with which it occurs — and not (!) 


on its interpretation within a modelling context or its “true 
nature” in a physical environment. 


4G.TH. FECHNER (1801-1887). 
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Ex. 1.7 (Logarithm). Recall from real analysis: For any given 
positive numbers a,x > 0, there is a unique number y = log, x 
such that 


a = ad = gS? 


where e = 2.718281828... is EULER’s!? number, the notation Inz = 
log, x is commonly used. In x is the so-called natural logarithm with 
the function derivative 


(nz)'=1/z forallz>0. 


Two logarithm functions log, x and log, x differ just by a multiplica- 
tive constant. Indeed, one has 


qiSat Sy= plose x = qlee a) log, x 
and hence 


log, © = (log, a)-log,x for all xz >0. 


Information. In SHANNON’s!% [42] fundamental theory of informa- 
tion, the parameter 


In(p) = — logy p 


is the quantity of information provided by an event FE that occurs 
with probability p. Note that the probability value p can be regained 
from the information quantity I2(p): 


p= glogsp — 9-la(p). 
This relationship shows that “probabilities” can be understood as 


numerical parameters that capture the amount of information (or 
lack of information) we have on the occurrence of events. 


127,, EULER (1707-1783). 
3C.E. SHANNON (1916-2001). 
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Entropy. The expected quantity of information provided by the 
family 


E={,...,En} 


of events with the probability distribution 7 = (pi,..., pn) is known 
as its entropy 


Hp(E) = = Yim (pr) --5 Pk 1089 Pr (8) 


where, by convention, one sets 0 - log,0 = 0. Again, it should be 
noticed: 


H»(€) just depends on the parameter vector 7 — and not on 


a real-world interpretation of €. 


REMARK 1.8. Entropy is also a fundamental notion in thermody- 
namics, where it serves, for example, to define the temperature of a 
system.'4 Physicists prefer to work with base e rather than base 2 
and thus with In x instead of log, x, i.e., with the accordingly scaled 
entropy 


Hl) = —~ Sopp In py = (In2) - Ho(7). 
k=1 


3. Systems 


A system is a physical, economic, or other entity that is in a certain 
state at any given moment. Denoting by G the collection of all 
possible states o, we identify the system with G. This is, of course, 
a very abstract definition. In practice, one will have to describe the 
system states in a way that is suitable for a concrete mathematical 
analysis. To get a first idea of what is meant, let us look at some 
examples. 


‘Chapter 7 relates the notion of temperature also to game-theoretic activities. 
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Chess. A system arises from a game of chess as follows: A state of 
chess is a particular configuration C' of the chess pieces on the chess 
board, together with the information which of the two players (“B” 
or “W”) is to draw next. If € is the collection of all possible chess 
configurations, a state could thus be described as a pair 


o=(C\p) withC € Cand pe {B,W}. 


In a similar way, a card game takes place in the context of a system 
whose states are the possible distributions of cards among the players 
together with the information which players are to move next. 


Economies. The model of an exchange economy involves a set N 
of agents and a set G of certain specified goods. A bundle for agent 
iz € N is a data vector 


b= (ba|G EG) ERY, 


where the component bg indicates that the bundle 6 comprises bg 
units of the good G € G. Denoting by B the set of all possible bundles, 
we can describe a state of the exchange economy by a data vector 


B=(68,|t¢N)€ BN 


that specifies each agent 2’s particular bundle 6; € B. 

Closely related is the description of the state of a general 
economy. One considers a set € of economic statistics E. Assuming 
that these statistics take numerical values eg at a given moment, the 
corresponding economic state is given by the data vector 


é=(en| BEE) eR’ 


having the statistical values eg as its components. 


Decisions. In a general decision system D, we are given a finite set 
N= ieee eee rin} 


of agents and assume that each agent i € N has to make a decision 
of a given type, i.e., we assume that each 7 has to designate an 
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element d; in its individual “decision set” D;. The joint decision of 
the members of NV is then a n-dimensional data vector 


d= (d;|i€ N) with n decision-valued components d; € D; 


and thus describes a decision state of the set N. In the context 
of game theory, decisions of agents often correspond to choices of 
strategies from certain strategy sets. 

Decision systems are ubiquitous. In the context of a traffic 
situation, for example, N can be a set of persons who want to travel 
from individual starting points to individual destinations. Suppose 
that each person i € N selects a path P; from a set P; of possible 
paths in order to do so. Then a state of the associated traffic system 
is a definite selection 7 of paths by the members of the group N and 
thus a data vector with path-valued components: 


nm = (Pili e N). 


3.1. Evolutions 


An evolution y of a system © over a time frame T is a function 
yp:TG 


with the interpretation: The system G is in the state y(t) at time t. 
While the notion of “time” is a philosophically unclear issue, let us 
keep things simple and understand by a time frame just a set T of 
real numbers. 

The time frames that are relevant in game-theoretic models are 
typically discrete in the sense that game-theoretic evolutions are 
observed at well-defined and well-separated time points t. So our 
time frames are of the type 


Day Sh etic Ste} 


Rather than speaking of the state o, = y(t,) of a system at time 
t = t, under the evolution y, it is often convenient to simply refer to 
the index n as the counter for the time elapsed. Hence an evolution 
y corresponds to a sequence of states: 


p <> 000102..-0n--. (On € G). 
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4. Games 


A game T involves a set N of agents (or players) and a system G6 
relative to which the game is played. A concrete game instance y 
starts with some initial state 09 € © and consists of a sequence of 
moves, i.e., of state transitions 


Ot — Ot4+1 


that are feasible according to the rules of I’. After ¢ steps, the system 
has evolved from state oo into a state o; in a sequence of (feasible) 
moves 


00 —* O01 > +++ +> Of-1 — Of. 


We refer to the associated sequence y= 0901 -+- 04-10% as the stage 
of T at time t and denote the set of all possible stages after ¢ steps 
by 


T. = {y% | % is a possible stage of T at time t}. (9) 


If the game instance y ends in stage jy = 0901--- 07, then o; is the 
final state of +. 


REMARK 1.9. It is important to note that there may be many finite 
state sequences y in © that are not feasible according to the rules of 
T and, therefore, are not stages of I. 


Abstract games. The preceding informal discussion indicates how 
a game can be defined from an abstract point of view: 
e A game [ on a system © is, by definition, a collection of 
finite state sequences y = 0901... 0% with the property 


o901°°'o1-10 €T = 0901: -1 ET. 


The members y € I are called the stages of T. 


Chess would thus be abstractly defined as the set of all possible finite 
sequences of legal chess moves. This set, however, is infinitely large 
and impossibly difficult to handle computationally. 
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In concrete practical situations, a game I is characterized by a 
set of rules that allow us to check whether a state sequence y is 
feasible for I’, i.e., belongs to that potentially huge set [. The rules 
typically involve also a set N of players (or agents) that “somehow” 
influence the evolution of a game by exerting certain actions and 
making certain choices at subsequent points in time t = 0,1, 2,.... 

Let us remain a bit vague on the precise mathematical meaning 
of “influence” at this point. It will become clear in special game 
contexts later. 

In an instance of chess, for example, one knows which of the 
players is to move at a given time t. This player can then move 
the system deterministically from the current state o; into a next 
state o441 according to the rules of chess. Many games, however, 
involve stochastic procedures (like rolling dice or shuffling cards) 
whose outcome is not known in advance and make it impossible for 
a player to select a desired subsequent state with certainty. 


REMARK 1.10. When a game starts in a state og at time t = 0, 
it is often not clear in what stage y it will end (or whether it ends 
at all). 


Objectives and utilities. The players in a game may have certain 
objectives according to which they try to influence the evolution of 
a game. A rigorous mathematical model requires these objectives to 
be clearly formulated in mathematical terms, of course. A typical 
example of such objectives is a family U of utility functions 


uiTl3R (GEN) 


which associate with each player i € N real numbers u;(y) € R as 
its utility value once the stage y € T is realized. 

Its expected utility is, of course, of importance for the strategic 
decision of a player in a game. We illustrate this with an example in 
a betting context. 


Ex. 1.8. Consider a single player with a capital of 100 euros in a 
situation where a bet can be placed on the outcome of a (0, 1)-valued 
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stochastic variable X with probabilities 

pH Pix jl) and g=]Prtix =0-=1T—@ 
Assume: 


e If the player invests f euros into the game and the event {X = 1} 
occurs, the player will receive 2f euros. In the event {X = 0} the 
investment f will be lost. 


Question: What is the optimal investment amount f* for the player? 
To answer it, observe that the player’s total portfolio after the bet is 
100 + f with probability p 
c=a2(f)= ; Fane 
100 — f with probability q 


For the sake of the example, suppose that the player has a utility 
function u(x) and wants to maximize the expected utility of x, that 
is the function 


o(f) =p-u(100 + f) + ¢q-u(100 — f). 
Let us consider two scenarios: 


(i) u(x) = x and hence the expected utility 


g(f) = p(100 + f) + q(100 — f) 


with derivative 
gf (f) =p—q=2p-1. 


If p < 1/2, the derivative is negative and, therefore, g(f) mono- 
tonically decreasing in f. Consequently f* = 0 would be the best 
decision. 


In the case p > 1/2, g(f) is monotonically increasing and, therefore, 
the optimal utility is to be expected from the full investment f* = 
100. 

(ii) u(x) = Inz and hence 


g(f) = pn(100 + f) + qIn(100 — f) 
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with the derivative 


(Ce _ 100p—q)—f 
IM) To0+ Ff 100—f 100004 f2 


In this case, g(f) increases monotonically until f = 100(p — q) and 


(0 < f < 100). 


decreases monotonically afterwards. So the best investment choice is 


f* =100(p—q) ifp>q. 


If p < q, we have 100(p — q) < 0. Hence f* = 0 would be the best 
investment choice. 


Nota BENE. The player in Ex. 1.8 with utility function u(x) = x 
risks a complete loss of the capital in the case p > 1/2 with 
probability q. 

A player with utility function u(#) = Inz will never experience 
a complete loss of the capital. 


Ex. 1.9. Analyze the betting problem in Ex. 1.8 for an investor with 
2 


utility function u(#) = 2°. 
REMARK 1.11 (Concavity). Utility functions which represent a 
gain are typically “concave”, which intuitively means that the 
marginal utility gain is higher when the reference quantity is small 
than when it is big. 


As an illustration, assume that u : (0,00) > R is a differentiable 
utility function. Then the derivative u(x) represents the marginal 
utility value at x. u is concave if the derivative function 


rru (x) 
decreases monotonically with a. 


The logarithm function f(x) = Inz has the strictly decreasing 
derivative f’(z) = 1/x and is thus an (important) example of a 
concave utility. 


Profit and cost. In a profit game the players 7 are assumed to aim 
at maximizing their utility u;. In a cost game one tries to minimize 
one’s utility to the best possible. 
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REMARK 1.12. The notions of profit and cost games are closely 
related: A profit game with the utilities u; is formally equivalent to 
a cost game with utilities c¢; = —u;. 


Terminology. A game with a set n of players is a so-called n-person 
game.’ The particular case of 2-person games is fundamental, as will 
be seen later. 


Decisions and strategies. In order to pursue its objective in an 
n-person game, an agent i may choose a strategy s; from a set 5; of 
possible strategies. The joint strategic choice 


8 = (s;|i € N) 


typically influences the evolution of the game. We illustrate the 
situation with a well-known game-theoretic puzzle: 


Ex. 1.10 (Prisoner’s dilemma). There are two agents A,B and 
the data matrix 


(Cie ’ ul) (uty, utd) 


(ud} ’ ub) (usb, uby) 


A and B play a game with these rules: 


(1) A chooses a row i and B a column j of U. 
(2) The choice (7,7) entails that A is “penalized” with the value ui 
and B with the value ub. 


This 2-person game has an initial state og and four other possible 
states (1,1), (1,2), (2,1), (2,2), which correspond to the four 
coordinate positions of U. 

The agents have to decide on strategies i,j € {1,2}. Their joint 
decision (7,7) will move the game from oo into the final state 0; = 
(i, 7). So the game ends at time ¢ = 1. The utility of player A is then 
the value us. B has the utility value ub. 


5 2 
lFven if the players are not real “persons”. 
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This game is usually understood as a cost game, i.e., A and B 
aim at minimizing their utilities. What should A and B do optimally? 


REMARK 1.13 (Prisoner’s dilemma). The utility matrix 
U in (10) yields a version of the so-called Prisoner’s dilemma, 
which is told as the story of two prisoners A and B who can 
either individually “confess” or “not confess” to the crime 
they are jointly accused of. Depending on their joint decision, 


they supposedly face prison terms as specified in U. Their 


“dilemma” is: 


e No matter what decisions are taken, at least one of the 
prisoners will feel that he has taken the wrong decision in 
the end. 
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Part 2 


2-Person Games 
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Chapter 2 


Combinatorial Games 


Games can always be understood as to involve two players that execute 
moves alternatingly. This aspect reveals a recursive character of games. 
The chapter takes a look at games that are guaranteed to end after a finite 


number of moves. Finite games are said to be combinatorial. Under the 
normal winning rule, combinatorial games have an algebraic structure and 
behave like generalized numbers. Game algebra allows one to explicitly 
compute winning strategies for nim games, for example. 


1. Alternating players 


Let [I be a game that is played on a system G and recall that [ 
represents the collection of all possible stages in an abstract sense. 
Assume that a concrete instance of I’ starts with the initial state a9 € 
©. Then we may imagine that the evolution of the game is caused 
by two “superplayers” that alternate with the following moves: 


The beginning player chooses a stage 71 = 0901 € I. 
The second player extends 7; to a stage yo = 090102 € I. 
Now it is again the turn of the first player to realize the 


next feasible stage y3 = 090203 € T and so on. 


The game stops if the player which would be next to move 
cannot find a feasible extension 741 € I of the current 
stage 7. 
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This point of view allows us to interpret the evolution of a game 
as the evolution of a so-called alternating 2-person game. For such a 
game A, we assume 


(Ao) There is a set G and two players LZ and R and an initial 
element Go € G. 


(A,) For every G € G, there are subsets GG and sag. 


The two sets G’ and G* in (A;) are the sets of options of the 
respective players relative to G. 
The rules of the alternating game JA are: 


(A3) The beginning player chooses an option G, relative to Go. 
Then the second player chooses an option G2 relative to 
G. Now the first player may select an option G3 relative 
to Gp and so on. 

(Aq) The game stops with G; if the player whose turn it is has 


no option relative to G; (i.e., the corresponding option set 
is empty). 


Ex. 2.1 (Chess). Chess is an obvious example of an alternating 
2-person game. Its stopping rule (A4) says that the game ends when 
a player’s king has been taken (“checkmate”). 


REMARK 2.1. While a chess game always starts with a move of the 
white player, notice that we have not specified whether L or R is the 
first player in the general definition of an altenating 2-person game. 
This lack of specification will offer the necessary flexibility in the 
recursive analysis of games below. 
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2. Recursiveness 


An alternating 2-person game A as above has a recursive structure: 


(R) A feasible move G > G" of a player reduces the current 


game to a new alternating 2-player game with initial 
element G’. 


To make this conceptually clear, we denote the options of the 
players L (“left”) and R (“right”) relative to G as 


@2(ChO eRe? (11) 


and think of G as the (recursive) description of a game that could 
possibly be reduced by L to a game Gr or by R to a game GR, 
depending on whose turn it is to make a move. 


3. Combinatorial games 
Consider an alternating 2-person game in its recursive form (11): 
CGHiG Ce ow AG Ge ssl: 


Denoting by |G| the maximal number of subsequent moves that are 
possible in G, we say that G is a combinatorial game if 


|G| < 00, 


i.e., if G is guaranteed to stop after a finite number of moves (no 
matter which player starts). Clearly, all the options GE and Ge of 
G must then be combinatorial games as well: 


IG|<co = > |GP,|GR| <|Gl|-1<.©. 


Ex. 2.2 (Chess). According to its standard rules, chess is not a 
combinatorial game because the players could move pieces back and 
forth and thus create a never ending sequence of moves. In practice, 
chess is played with an additional rule that ensures finiteness and 


36 2. Combinatorial Games 


thus makes it combinatorial (in the sense above). The use of a timing 
clock, for example, limits the the number of moves. 


Ex. 2.3 (Nim). The nim game G = G(M,..., Nx) has two 
alternating players and starts with the initial configuration of 
a collection of & finite and pairwise disjoint sets N1,...,Nz. A 
move of a player is: 


e Select one of these sets, say N;, and remove one or more 
of the elements from Nj. 


Clearly, one has 
|G(Ni,...,Ne)| < |Ni| +... + |Ne| < co. 


So nim is a combinatorial game. 


“A popular version of nim starts from four sets Ni, No, N3, Na of pieces 
(pebbles or matches, etc.) with |Ni| = 1,|Ne2| = 3,|N3| = 5 and |Na| = 7 
elements. 


Ex. 2.4 (Frogs). Having fixed numbers n and k, two frogs L and 
R sit n positions apart. A move of a frog consists in taking a leap of 
at least 1 but not more than k positions toward the other frog: 


(L) > ee e---e 00 «(R) 


The frogs are not allowed to jump over each other. Obviously, the 
game ends after at most n moves. 


REMARK 2.2. The game of frogs in Ex. 2.4 can be understood as 
a nim game with an additional move restriction. Initially, there is a 
set N with n elements (which correspond to the positions separating 
the frogs). A player must remove at least 1 but not more than k 
elements. 


Creation of combinatorial games. The class i of all combinato- 
rial games can be created systematically. We first observe that there 
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is exactly one combinatorial game G with |G| = 0, namely the game 
O=% | +t 
in which no player has an option to move. Recall furthermore that all 
options G¥ and G* of a game G with |G| = t must satisfy |G”| < t-1 


and |G®| < t— 1. So we can imagine that % is “created” in a never 
ending process from day to day: 


Day 0: The game O = {- | -} is created and yields Ro = {O}. 


Day 1: The games {O|-}, {- | O},{O | O} are created and one 
obtains the class 


Ri = {0,{O]-}, {| OF, 10 | OFF 


of all combinatorial games G with |G| < 1. 


Day 2: The creation of the class 8» of those combinatorial 
games with options in 9; is completed. These include the 
games already in 4; and the new games 


CHO t LLOldE Ct LOW... 
{O|{O|-}}, 10 | {Ol}, {0 | {| OFF. 

{O, {- | OF{O]-33, {0,10 | F110, {0 | Ht. 
{0, {| OF{O] Fh... 


Day t: The class SR; of all those combinatorial games G with 
options in Siy_1 is created. 


So one has Ro C My C... CHC... and 


R= Ro UR U...UR VU... = lim Re. 


Ex. 2.5. The number of combinatorial games grows rapidly: 


(1) List all the combinatorial games in Ro. 
(2) Argue that many more than 6000 combinatorial games exist at 
the end of Day 3 (see Ex. 2.6). 
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Ex. 2.6. Show that r; = |9;| grows super-exponentially fast: 
hee (t = Lee) 


(Hint: A finite set S with n = |S| elements admits 2” subsets.) 


4. Winning strategies 


A combinatorial game is started with either LZ or R making the first 
move. This determines the first player. The other player is the second 
player. The normal winning rule for an alternating 2-person games is: 


(NR) If a player 7 € {L, R} cannot move, player 7 has lost and 


the other player is declared the winner. 


Chess matches, for example, are played under the normal rule: 
A loss of the king means a loss of the match (see Ex. 2.1). 


REMARK 2.3 (Misére). The misére rule declares the player with 
no move to be the winner of the game. 


A winning strategy for player i is a move (option) selection rule 
for 7 that ensures 7 to end as the winner. 


THEOREM 2.1. In any combinatorial game G, an overall 


winning strategy exists for either the first or the second player. 


Proof. We prove the theorem by mathematical induction on t = |G|. 
In the case t = 0, we have 


G=0={-|}. 


Because the first player has no move in O, the second player is auto- 
matically the winner in normal play and hence has a winning strategy 
trivially guaranteed. Under the misére rule, the first player wins. 
Suppose now ¢ > 1 and that the theorem is true for all games 
that were created on Day t — 1 or before. Consider the first player 
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in G and assume that it is R. (The argument for L would go exactly 
the same way!) 

If R has no option, L is the declared winner in normal play while 
R is the declared the winner in misére play. Either way, G has a 
guaranteed winner. 

If options G®” exist, the induction hypothesis says that each of 
R’s options leads to a situation in which either the first or the second 
player would have a winning strategy. 

If there is (at least) one option G® with the second player as the 
winner, R can take this option and win as the second player in G®. 

On the other hand, if all of R’s options have their first player as 
the winner, there is nothing R can do to prevent L from winning. So 
the originally second player L has a guaranteed overall strategy to 


win the game. 


Note that the proof of Theorem 2.1 is constructive in the 
following sense: 


Player i marks by v(G’) = +1 all the options G* in G 
that would have 2 winning as the then second player and 


sets v(G") = —1 otherwise. 


Player i follows the strategy to move to an option with 
the highest v-value. 

Provided a winning strategy exists at all for 7, strategy 
(2) is a winning strategy for 7. 


The reader must be cautioned, however. The concrete computa- 
tion of a winning strategy may be a very difficult task in real life. 


Ex. 2.7 (DE BRUIJN’s game). Two players choose a natural 
number n > 1 and write down all n numbers 


13:2 ,3)2355%— 1: 


A move of a player consists in selecting one of the numbers still 
present and erasing it together with all its (proper or improper) 
divisors. 
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Note that a winning strategy exists for the first player in normal 
play. Indeed, if it existed for the second player, the first player could 
simply erase “1” on the first move and afterwards (being now the 
second player) follow that strategy and win. Alas, no practically 
efficient method for the computation of a winning strategy is 
known. 


REMARK 2.4. If chess is played with a finiteness rule, then a 
winning strategy exists for one of the two players. Currently, however, 
it is not known what it looks like. It is not even known which player 
is the potentially guaranteed winner. 


Playing in practice. While winning strategies can be computed in 
principle (see the proof of Theorem 2.1), the combinatorial structure 
of many games is so complex that even today’s computers cannot 
perform the computation efficiently. 

In practice, a player 7 will proceed according to the following 
v-greedy strategy!: 


(vg; ) Assign a quality estimate v(G") € R to all the options G* 


and move to an option with a highest v-value. 


A quality estimate v is not necessarily completely pre-defined 
by the game in absolute terms but may reflect previous experience 
and other considerations. Once quality measures are accepted as 
“reasonable”, it is perhaps natural to expect that the game would 
evolve according to greedy strategies relative to these measures. 


Ex. 2.8. A popular rule of thumb evaluates the quality of a chess 
configuration o for a player W, say, by assigning a numerical weight v 


'Also chess computer programs follow this idea. 
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to the white pieces on the board. For example: 


Where v(c) is the total weight of the white pieces, a v-greedy player 
W would choose a move to a configuration o with a maximal value 
v(o). (Player B can, of course, evaluate the black pieces similarly.) 


5. Algebra of games 


For the rest of the chapter we will (unless explicitly said otherwise) 
assume: 


e The combinatorial games under consideration are played with the 
normal winning rule. 


The set 9% of combinatorial games carries an algebraic structure 
which allows us to do computations with games as generalized 
numbers. This section wants to give a short sketch of the idea.” 


Negation. We first define the negation (—G) for the game 
GH=4E". Goo GY Gr 2 ent 


as the game that arises from G when the players L and R interchange 
their roles: L becomes the “right” and R the “left” player. 


?(Much) more can be found in the highly recommended treatise of Conway [7]. 
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So we obtain the negated games recursively as 
—O=O 
GSieGt =Co... | SG) =6Gh uy 4E6 £0: 


Also —G is a combinatorial game and one has the straightforward 
algebraic rule 


—(-—G) =G. 
Addition. The sum G+ H of the games G and H is the game in 
which a player 7 € {L, R} may choose to play either on G or on H. 


This means that i chooses an option G’ in G or an option H* in H 
and accordingly reduces the game 


either to G'+H orto G+H". 
The reader is invited to verify the further algebraic rules: 


GaH=Hee 
(GAMeRSGLHEH) 
C+0=6. 


Moreover, we write 


G-H=G+(-4#). 


Ex. 2.9. The second player wins G— G (= G+ (-G)) in 
normal play with the obvious strategy: 


e Imitate every move of the first player: 


When the first player chooses the option G’ in G, the 
second player will answer with the option (—G") in (—G), 
etc. 


5.1. Congruent games 


Motivated by Ex. 2.9, we say that combinatorial games G and H are 
congruent (notation: “G = H”) if 


(C) G — H can be won by the second player (in normal play). 


5. Algebra of games 43 
In particular, G = O means that the second player has a winning 


strategy for G. 


THEOREM 2.2 (Congruence Theorem). For all G,H, 
K ER, one has: 


(a) IfG=H, then H=G. 
(b) G=H, thnG+K=H+K. 
(c) [fG=H andH=K, thenG=K. 


Proof. The verification of the commutativity rule (a) is left to the 
reader. To see that (b) is true, we consider the game 


M=(G+K)-(H+K) 
=G+K-H-K 
=(G—H)+(K-K). 


The game K — K can always be won by the second player (Ex. 2.9). 
Hence, if the second player can win G — H, then clearly M as well: 


e It suffices for the second player to apply the respective 
winning strategies toG— H andtoK—K. 


The proof of the transitivity rule (c) is similar. By assumption, 
the game 


T=(G—K)+(-H+4H)=(G- 4H)+(H-K) 


can be won by the second player. We must show that the second 
player can therefore win G— K. 

Suppose to the contrary that G— K # O were true and that 
the game G — K could be won by the first player. Then the first 
player could win T by beginning with a winning move in G— K and 
continuing with the win strategy whenever the second player moves 
in G — K. If the second player moves in K — K, the first player 
becomes second there and thus is assured to win on K — K! So the 
first player would win 7, which would contradict the assumption 
however. 
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Hence we conclude that G— K = O must hold. 


Congruence classes. For any G € Ki, the class of congruent games 
is 


[G] = {H €R|G = H}. 


Theorem 2.2 says that addition and subtraction can be meaning- 
fully defined for congruence classes: 


(G]+|H]=|G+H] and [G|-[H] =(|(G- dq]. 
In particular, we obtain the familiar algebraic rule 
[G] — [G] = [|G - G] = [O], 


where [O] is the class of all combinatorial games that are won by the 
second player. Hence we can re-cast the optimal strategy for a player 
(under the normality rule): 


e Winning strategy: 


Make a move G -—> G’ to an option G’ € [O]. 


5.2. Strategic equivalence 


Say that the combinatorial games G and H are strategically equiva- 
lent (denoted “G ~ H”) if one of the following statements is true: 


(SE,) G and H can be won by the first player (7.e., G#O # H). 
(SE) G and H can be won by the second player (7.e., G = O = H). 


THEOREM 2.3 (Strategic equivalence). Congruent games 
G,H € are strategically equivalent, i.e., 


G=H => Gwd. 


Proof. We claim that strategically non-equivalent games G and H 
cannot be congruent. 
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So assume, for example, that the first player wins G (1.e.,G # O), 
and the second player wins H (i.e., H = O and hence (—H) = O). 
We will argue that the first player has a winning strategy for G— H, 
which means G # H. 

Indeed, the first player can begin with a winning strategy on G. 
Once the second player moves on (—#Z), the first player, being now 
the second player on (—H), wins there. Thus an overall victory is 
guaranteed for the first player. 


6. Impartial games 


A combinatorial game G' is said to be impartial (or neutral) if both 
players have the same options. The formal definition is recursive: 


e O={.|-} is impartial. 
e G={A,B,...,T | A, B,...} is impartial if all the options 


A, B,...,T are impartial. 


Notice the following rules for impartial games G and H: 


(1) G=-G and hence G+G=G-—Ge [O]. 


(2) G+ 4H is impartial. 


Ex. 2.10. Show that the frog game of Ex. 2.4 is impartial. 


Nim is the prototypical impartial game (as the SPRAGUE—GRUNDY 
Theorem 2.4 will show below). 

To formalize this claim, we use the notation *n for a nim game 
relative to just one single set Ny with n = |Nj| elements. The options 
of *n are the nim games 

*0, *1,...,*(n — 1). 
Moreover, 


G = ny + Ng +-°+ + *NE 


is the nim game described in Ex. 2.3 with k piles of sizes 
N1,12,...,Mk.- 
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We now define the mex? of numbers a,b,c...,t as the smallest 
natural number g that equals none of the numbers a, b,c,...,t: 


mex{a,b,c,...,t} = min{g € No | g ¢ {a,b,c,..., t}}, (12) 


The crucial observation is stated in Lemma 2.1. 


LEMMA 2.1. For any finitely many numbers a,b,c,...,t € 
No, one has 


C= ADE Caer | PEE DLO oe = PIMOK OD Car iaet 


ie., the impartial game G with the nim options xa, xb, *c, 
...,*t is congruent with the simple nim game *m with 


fe LOX UU Ones at 


Proof. In view of «m = —*m, we must show: G+ *m = O, 1.e., 
the second player wins G + *m. Indeed, if the first player chooses an 
option *7 from 


xm = {40, #1)... 40n-— 1}, 


then the second player can choose *j from G (which must exist 
because of the definition of m as the minimal excluded number) and 
continue to win «7 + *7 as the second player. 

If the first player selects an option from G, say xa, we distinguish 
two cases. If a > m then the second player reduces *a to *m and 
wins. If a < m, then the second player can reduce *m to *a and win. 
(Note that a = m is impossible by the definition of mex.) 


3 “Minimal excluded”. 
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THEOREM 2.4 (SPRAGUE—GRUNDY). Every impartial com- 
binatorial game 


CEO 6 ee EOS. ce 


is congruent to a unique nim game of type xm. m is the so- 
called GRUNDY number of G and denoted by G(G). 


G(G) of can be computed recursively from the GRUNDY num- 
bers of the options: 


G(G) = mex{G(A),G(B),G(C),...,G(T)}. (13) 


Proof. We prove the theorem by induction on |G| and note G = O 
if |G| = 0. By induction, we now assume that the theorem is true for 
all options of G, i.e., A= *a, B = *b etc. with a = G(A), b=G(B), 
etc. 

Hence we can argue G = *m = *G(G) exactly as in the proof of 
Lemma 2.1. G cannot be congruent with another nim game +k since 
(as Ex. 2.11 below shows): 


akhe=*em = k=m. 


Ex. 2.11. Show for all natural numbers k and n: 
aheen — ke=n. 


Ex. 2.12 (GRUNDY number of frogs). Let F(n,k) be the 
(impartial) frog game of Ex. 2.4 and G(n,k) its GRUNDY number. 
For k = 3, F(n,k) has the options 


F(n — 1,3), F(n — 2,3), F(n — 3,3). 
So the associated GRUNDY number G(n,3) has the recursion 


G(n,3) = mex{G(n — 1,3), G(n — 2,3),G(n — 3)}. 
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Clearly, G(0,3) = 0,G(1,3) = 1 and G(2,3) = 2. The recursion then 
produces the subsequent GRUNDY numbers: 


— 
Nlo 


The second player wins the nim game *m if and only if m = 0. So 
the first player can win exactly the impartial games G with GRUNDY 
number G(G) # 0. In general, we note: 


e Winning strategy for impartial games: 


Make a move G — G’ to an option G’ 
with a GRUNDY number G(G’) = 0. 


6.1. Sums of GRUNDY numbers 


If G and H are impartial games with GRUNDY numbers m = G(G) 
and n = G(H), the GRUNDY number of their sum is 


G(G+ H) = G(«n + xm) 


Indeed, if G = *m and H = *n, then G+ H = *m+ *n must hold.4 
For the study of sums, we may therefore restrict ourselves to nim 
games. Moreover, the fundamental property 


G(G + G) =G(*n + *n) =G(O) =0 
suggests to study sums in the context of binary algebra. 


Binary algebra. Recall that every natural number n has a unique 
binary representation in terms of powers of 2, 


[oe) 
n= ) aj2, 
j=0 


*Recall Theorem 2.2! 


6. Impartial games 49 


with binary coefficients a; € {0,1}. We define binary addition of 0 
and 1 (as in Section 1.2.3) according to the rules 


060=0=1061 and 0G1=1=1060 


and extend it to natural numbers: 


2 . ve . &. . 
dj? | & | 7 852? | = So(aj @ By)’. 
j=0 j=0 j=0 


REMARK 2.5. Notice that the binary representation of a natural 
number has only finitely many non-zero summands. Indeed, a; = 0 
must hold for all 7 > logs n if 


[o-e) 
ve a? with a; € {0, 1}. 
j=0 
Ex. 2.13. Show for the binary addition of natural numbers m,n, k: 


nom = m@en 
noe(m@k) = (nOom)ek 
neomek=0—neOme= k. 


The sum theorem. We consider nim games with three piles of n,m 
and k objects, z.e., sums of three single nim games *n, *m, and xk. 


LEMMA 2.2. For all numbers n,m,k € No, one has: 


(1) Ifn@®me@k F0, then the first player wins *n+*m-+ xk. 
(2) Ifn@mek = 0, then the second player wins *n+*m-+xk. 


Proof. We prove the lemma by induction on n+m-+ & and note 
that the statements (1) and (2) are obviously true in the case 


n+m+k=0. 


By induction, we now assume that the lemma is true for all 
n',m',k’ € No such that 


nt+m +k <nt+tme+k. 
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We must then show that the lemma holds for n,m, k with the binary 
representations 


(oe) (oe) [oe) 
n= S 5 02%, m= S > 32, k= yee 
j=0 j=0 j=0 


In the case (1) with n@m@k ¥ 0, there must be at least one j such 
that 


a; © 8; By; = 1. 


Let J be the largest such index 7. Two of these coefficients a7, 87, 77 
must be equal and the third one must have value 1. So suppose 
ay = 6; and yz; = 1, for example, which implies 


nom<k and n+m4+(n@m)<n+m+tk. 


Let k' =n @m. We claim that the first player can win by reducing 
*k to *k’. Indeed, the induction hypothesis says that the lemma is 
true for n,m, k’. Since 


nomek' =n@menem=d, 


property (2) guarantees a winning strategy for the second player in 
the reduced nim game 


an + am + xk’. 


But the latter is the originally first player! So statement (1) is found 
to be true. 

In case (2), when n 6m @k = 0, the first player must make a 
move on one of the three piles. Let us say that *n is reduced to *n’. 
Because n = m@ k, we have 


n’Am@k andtherefore n’' OmMOkFO. 


Because the lemma is assumed to be true for n’,m,k, statement (1) 
guarantees a winning strategy for the first player in the reduced game 


xn’ +«mx*k, 


which is the originally second player. 
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THEOREM 2.5 (Sums of impartial games). For any imp- 
artial combinatorial games G and H, one has 


G(G + H) =G(G) © G(H). 


Proof. Let n = G(G) and m= G(H) andk =n@m. Thenn@m+ 
k; = 0 holds. So Lemma 2.2 says that the second player wins 


n*k+*xmx«(n@m), 
which yields 


G+ H=*n+x«m=+*(n@m). 


Consequently, n @m must be the GRUNDY number of G+ H. 
We illustrate Theorem 2.5 with the nim game 


G=*lt+x«3*«+%*5+%*7 


of four piles with 1,3,5 and 7 objects respectively. The binary 
representations of the pile sizes are 
Pao? 
Yeas ey es ee 
5=1-2°+0-2'+1-2? 
fis 2? eee Ot 


So the GRUNDY number of G is 
G(xl +43 * + %5+%*7)=1030507=0. 
Hence G can be won by the second player in normal play. 


Ex. 2.14. Suppose that the first player removes 3 objects from the 
pile of size 7 in G = *1 + «3 * + *5 + *7. How should the second 
player respond? 


Ex. 2.15. There is a pile of 10 red and another pile of 10 black 
pebbles. Two players move alternatingly with the following options: 
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e EITHER: take at least 1 but not more than 3 of the red pebbles 
OR: take at least 1 but not more than 2 of the black pebbles. 


Which of the players has a winning strategy in normal play? (Hint: 
Compute the GRUNDY numbers for the red and black piles separately 
(as in Ex. 2.4) and apply Theorem 2.5.) 


Chapter 3 


Zero-Sum Games 


Zero-sum games abstract the model of combinatorial games. Fundamental 
examples arise naturally as LAGRANGE games from mathematical opti- 
mization problems and thus furnish an important link between game 
theory and mathematical optimization theory. In particular, strategic 


equilibria in such games correspond to optimal solutions of optimization 
problems. Conversely, mathematical optimization techniques are impor- 
tant tools for the analysis of game-theoretic situations. 


As in the previous chapter, we consider games I’ with 2 agents (or 
players). However, we shift the viewpoint and no longer assume 
players taking turns in moving a system from one state into another. 
Instead, we assume that the players decide on strategies according 
to which they play the game. 

More precisely, the players are assumed to have sets X and Y 
of possible strategies (or decisions, or actions, etc.) at their disposal. 
Hence a “state” of the game under this new perspective is a pair 
(x,y) of strategies x € X and y € Y.1 In fact, we consider the set 


S={(x,y)|eeX,yeV}U {oo} 


of joint strategy choices as the system underlying the game I’, where 
oo is just an initial state, which we may attach formally to the set 
X x Y of joint strategies. We therefore refer to the one player as the 
x-player and to the other as the y-player. 


' As in the Prisoner’s dilemma (Ex. 1.10). 
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T is a zero-sum game if there is a function 
U:XxY->R 


that encodes the utility of the strategic choice (x, y) in the sense that 
the utility values of the individual players add up to zero: 


(1) u(x, y) = U(a, y) is the gain of the x-player; 
2) ua(z,y) = —U(z, y) is the gain of the y-player. 


—— 


It follows that the two players have opposing goals: 


(X) The x-player wants to choose x € X as to maximize U(x, y). 
(Y) The y-player wants to choose y € Y as to minimize U(x, y). 


We denote the corresponding zero-sum game by T = T(X,Y,U). 


Ex. 3.1. A combinatorial game with respective strategy sets X and 
Y for the two players is, in principle, a zero-sum game with the utility 
function 


+1 if x is a winning strategy for the X-player 
U(x,y) =< —1 if y is a winning strategy for the Y-player 
0 otherwise. 


Ex. 3.2. The Prisoner’s dilemma is a 2-person game. However, the 


utilities of Ex. 1.10 do not lead to a zero-sum game. 


1. Matrix games 


In the case of finite strategy sets, say X = {1,...,m} and Y = 


{1,...,n}, a function U : X x Y > R can be presented in matrix 
form: 
U11 U21 oe. Uln 
ve U21 U22 =... U2n 2 Rn 
Um1 Um2 +--+ Umn 


The associated matrix game is the zero-sum game [ = (X,Y,U) 
where the x-player chooses a row 7 and the y-player a column j. 


2. Equilibria 55 


This joint selection (7,7) has the utility value u;; for the row player 
and the value (—u,;;) for the column player. As an example, consider 
the game with the utility matrix 


ee ie | (14) 


In the example (14), there is no obvious overall “optimal” choice 
of strategies. No matter what row 7 and column j are selected by 
the players, one of the players will find? that the other choice would 
have been more profitable. In this sense, this matrix game has no 
“solution”. 

Before pursuing this point further, we will introduce a general 
concept for the solution of a zero-sum game in terms of an equilibrium 
between both players. 


2. Equilibria 


Let us assume that both players in the zero-sum game I’ = (X, Y,U) 
are risk avoiding and want to ensure themselves optimally against 
the worst case. So they consider the worst case functions 


Uile)= min u(x, ¥) €E RU {-oo} 


15 
Ur{y) = maxu(e,y) € RU {+00}. _ 
The «-player thus faces the primal problem 
— 1 1 
max Uj; (x) ne U(z,y), (16) 
while the y-player is to solve the dual problem 
min Uo(y) = minmax U(z,y). (17) 


yeY yEY rex 


?In hindsight! 
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From the definition, one immediately deduces for any « € X 


and y € Y the primal-dual inequality: 
Ui (x) < U(z,y) < Valy) (18) 


We say that (x*,y*) € X x Y is an equilibrium of the game I if 
it yields the equality 


Ui(2") = Ua(y"), 


i.e., if the primal-dual inequality, in fact, attains equality: 


Tiaxemnel) (ey — a ye ee ap) (19) 


xEX yEeY yEY rex 


In the equilibrium (x*,y*), none of the risk avoiding players has 
an incentive? to deviate from the chosen strategy. In this sense, 
equilibria represent optimal strategies for risk avoiding players. 


Ex. 3.3. Determine the best worst-case strategies for the two 
players in the matrix game with the matrix U of (14) and show 
that the game has no equilibrium. 

Give furthermore an example of a matrix game that possesses at 
least one equilibrium. 


Finding equilibria. If the strategy sets X and Y are finite and 
hence the zero-sum game [T = (X,Y,U) is a matrix game, the 
question whether an equilibrium exists, can — in principle — be 
answered in finite time by a simple procedure: 


e Check each strategy pair (a*,y*) € X x Y for the pro- 


perty (19). 


3In hindsight. 


3. Convex zero-sum games 57 


If X and Y are infinite, usually the existence of equilibria can only 
be decided if the function U : X x Y > R has special properties. 
From a theoretical point of view, the notion of convexity is very 
helpful and important. 


3. Convex zero-sum games 


Recall* that a conver combination of points 71,...,7% € R” is a 
linear combination 


k k 
t= oe 4x; with coefficients A; > 0 such that MS Ay = 1. 
j=1 i=1 


An important interpretation of © is based on the observation that 
the coefficient vector X = (Ai,...,A,) of a convex combination is a 
probability distribution: 


e If a point x; is selected from the set {x,...,2%} with 


probability 4;, then the components of the convex com- 
bination % are exactly the expected component values of 
the stochastically selected point. 


Another way of looking at & is: 


e If weights of size A; are placed on the points x;, then Z is 
their center of gravity. 


A set X C R” is convex if X contains all convex combinations of 
all possible finite subsets {x1,...,x7,} CX. 


Ex. 3.4. Let S = {s1,...,5m} be an arbitrary set with m > 1 


elements. Show that the set 5 of all probability distributions \ on S 
forms a compact convex subset of R”. 


“See also Appendix A.2 for more details. 
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A function f : X — R is convex (or conver up) if X is a convex 
subset of some coordinate space R” and for every 71,...,%, € X and 
probability distribution \ = (A1,...,A,), one has 


FO +... + Agee) S Arf (ei) +... + ARF (TE). 


f is concave (or conver down) if g = —f is convex (up). 
With this terminology, we say that the zero-sum game [ = 
(X,Y,U) is convex if 


(1) X and Y are non-empty convex strategy sets; 
(2) the utility U: X x Y > R is such that 


(a) for every y € Y, the map +> U(z, y) is concave. 
(b) for every x € X, the map y+> U(z, y) is convex. 


The main theorem on general convex zero-sum-games guarantees 
the existence of at least one equilibrium in the case of compact 
strategy sets: 


THEOREM 3.1. A convex zero-sum game T = (X,Y,U) with 


compact strategy sets X and Y and a continuous utility U 
admits a strategic equilibrium (a*,y*)E Xx Y. 


Proof. Since X and Y are convex and compact sets, so is the set 
Z = X x Y and hence also the set Z x Z. 
Consider the continuous function G : Z x Z > R with the values 


G((a',y'), (z,y)) = U(2,y') _ U(a',y). 


Since U is concave in the first variable x and (—U) concave in the 
second variable y, we find that G is concave in the second variable 
(x,y). Hence Corollary A.1 (Appendix) allows us to deduce the 
existence of an element (a*,y*) € Z that satisfies 


0 = G((x",y*), (@",y")) 
= G((2",y"), (@, y)) = U(a,y") — Ua", y) 
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for all (x,y) € Z and hence the inequality 
U(z,y*) < U(x"*,y) forallae X andallyeY. 


This shows that x* is the best strategy for the x-player if the y-player 
chooses y* € Y. Similarly, y* can be proved to be optimal against 
x*. In other words, (a*, y*) is an equilibrium of (X,Y,U). 

Theorem 3.1 has important consequences not only in game theory 
but also in the theory of mathematical optimization in general, which 
we will sketch in more details in Section 4 below. To illustrate the 
situation, let us first look at the special case of randomizing the 
strategic decisions in zero-sum matrix games. 


3.1. Randomized matrix games 


Recall from Ex. 5.2 that a zero-sum game T = (X,Y,U) with finite 
strategy sets X and Y does not necessarily admit an equilibrium. 

Suppose the players randomize the choice of their respective 
strategies. That is to say, the x-player decides on a probability 
distribution on X and chooses an i € X with probability 7;. 
Similarly, the y-player chooses a probability distribution 7 on Y 
and selects 7 € Y with probability 7;. Then the x-player’s expected 
gain is 


(i UG, y) = Soo uiyBi;, 


tEX FEY 


where the u;; are the coefficients of the utility matrix U. 

So we arrive at a zero-sum game I = (X,Y,U), where X is the 
set of probability distributions on X and Y the set of probability 
distributions on Y. X and Y are compact convex sets (cf. Ex. 3.4). 

Moreover, the function U : X x Y — R is linear, and thus 
continuous and both concave and convex in both components. 

It follows that T is a convex game that satisfies the hypothesis 
of Theorem 3.1. Therefore, [ admits an equilibrium. This proves the 
VON NEUMANN’s Theorem: 
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THEOREM 3.2 (VON NEUMANN [32]). Let U € R™*" be 
an arbitrary matric with coefficients uj;. Then there exist 
a* € X and y* € Y such that 


max min x UijLiYj = y Wij li Y; = min max 2 UsjLiy;- 


1l<i<m l<y<ms l<j<n 1<i<m 


where X is the set of all probability distributions on {1,...,m 
and Y the set of all probability distributions on {1,...,n 


3.2. Computational aspects 


While it is generally not easy to compute equilibria in zero-sum 
games, the task becomes tractable for randomized matrix games. 
Consider, for example, the two sets X = {1,...,m} and Y = 
{1,...,7} and the utility matrix 


UI{I1 U21 eee Ulin 
uy U21 U22 Sis U2n Rmxn 
= E 
| Um1 Um2 -:-- bec 


For the probability distributions x € X and y € Y, the expected 
utility for the «-player is 


m n n m 
zy) = Ss" So uajtiy; = Soy bs ws . 
i=1 j=l j= i=l 


The worst case for the row player occurs when the column player 
selects a probability distribution that puts the full weight 1 on k € Y 
such that 


Ya = = in| So Es ae anh = Ui (2). 
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Hence 


m 
mia: Uy (a) = “tax: <2] 2 ) tee ees tor all os sy 
rex zER vex j=1 


(20) 


Similarly, the worst case for the column player it attained when the 
row player places the full probability weight 1 onto @€ X such that 


n n 
S/ uejyj = max ela haw = Uo(y). 
j=l j=l 
This yields 


n 
minU2(y) = min _ wlw> So uijy; for all 7 =1,...,n 
yeY weR,yeY vai 


(21) 


This analysis shows: 


PROPOSITION 3.1. Jf (z*,a*) is an optimal solution of (20) 
and (w*,y*) and optimal solution of (21), then 


(1) (a*,y*) is an equilibrium of T = (X,Y,U). 


(2) 2* = max U;(x) = min U2(y) = w". 
cEX yeY 


REMARK 3.1. As further outlined in Section 4.5 below, the opti- 
mization problems (20) and (21) are linear programs that are dual 
to each other. They can be solved very efficiently in practice. For 
explicit solution algorithms, the interested reader may consult the 
standard literature on mathematical optimization.® 


e.g., FAIGLE et al. [17]. 
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4. LAGRANGE games 


The analysis of zero-sum games is closely connected with a fun- 
damental technique in mathematical optimization. A very general 
formulation of an optimization problem is 


max f (2), 


LeEF 


where F could be any set and f : F > R an arbitrary objective 
function. In our context, however, we will look at more concretely 
specified problems and understand by a mathematical optimization 
problem a problem of the form 


> 
max fie) ssuch that) gia j= 


where X is a nonempty subset of some coordinate space R” with an 
objective function f: X > R. 

The vector valued function g: X > R™ is a restriction function 
and combines m real-valued restriction functions g; : X — R as its 
components. The set of feasible solutions of (22) is 


F={e €X. |\oi(a) > 0 for allelic. 


REMARK 3.2. The model (22) formulates an optimization problem 
as a maximization problem. Minimization problems can, of course, 
also be formulated within this model because of 


min f(#) = — max f(x) with the objective f(x) = —f (a). 
LeEF LEF 


The optimization problem (22) defines a zero-sum game A = 
(X,R?,L) with the so-called LAGRANGE function 
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as its utility. We refer to A as a LAGRANGE game.® The worst-case 
utility functions of the two players in A are: 


Ly(x) = min L(x,y) = f(x) + min 2 vigila) 


y20 


is 
=max L(x,y) = max f(x +S wale 


Ex. 3.5 (Convex LAGRANGE games). If X is convex and the 
objective function f : X — R as well as the restriction functions 
gi: X — R are concave, then the LAGRANGE game A = (X,R‘, L) 
is a convex zero-sum game. Indeed, L(x, y) is concave in x for every 
fixed y > O and linear in y for every fixed x € X. Since linear 
functions are in particular convex, the game A is convex. 


4.1. Complementary slackness 


The choice of an element x € X with at least one restriction 
violation g;(a) < 0 would allow the y-player in the LAGRANGE game 
A = (X,R", L) to increase its utility value infinitely with the choice 
y; © co. So the risk avoiding x-player will always try to select a 
feasible zx. 

On the other hand, if all of the feasibility requirements g;(x) > 0 
are satisfied, the best the y-player can do is the selection of y € R"? 
such that the so-called complementary slackness condition 


S- vigi(x) =y"'g(z)=0 andhence L(z,y) = 


is met. Consequently, one finds: 


®The idea goes back to J.-L. LAGRANGE (1736-1813). 
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The primal LAGRANGE problem is identical with the original 
problem: 


in L = L(x) = (25 
als (x,y) = max In(x) = max f(z) (25) 


LEMMA 3.1. If (x*,y*) is an equilibrium of the LAGRANGE 
game A, then x* is an optimal solution of problem (22). 


Proof. For every feasible x € F, y* > 0 yields (y*)" g(x) > 0 and, 
therefore, 


f(w*) = Ly(a*) = Loly*) = f(x) + (y*)" 9(@) = f(@). 


So «* is optimal. 


4.2. The KKT-conditions 


Lemma 3.1 indicates the importance of being able to identify 
equilibria in LAGRANGE games. In order to establish necessary con- 
ditions, 7.e., conditions which candidates for equilibria must satisfy, 
we impose further assumptions on problem (22): 


(1) X CR” is a convex set, i.e., X contains with every x, x’ also the 
whole line segment 


x, a’] = {a + A(a’ — 2) |0<A< 1}. 


(2) The functions f and g; in (22) have continuous partial derivatives 
Of(x)/Ox; and Og;(x)/Ox; for all j = 1,...,n. 


It follows that also the partial derivatives of the LAGRANGE 
function L exist. So the marginal change of L into the direction d of 
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the x-variables is 


ViL(a,y)d = VF (a at ava a)d 
i=1 
” ais ails 
= +E Se 
Soe 


i=1 j=1 


REMARK 3.3 (JACOBI matrix). The (mx n) matrix Dg(x) hav- 
ing as coefficients the partial derivatives 


Dg(2)ij = Ogi(x)/0x; 


of a function g : R” > R™ is known as a functional or JACOBI" 
matrix. It provides a compact matrix notation for the marginal 
change of the Lagrange function: 


VeL(a,y)d = Vf (x) + y? Dg(a)d. 


LEMMA 3.2 (KKT-conditions). The pair (x,y) € X x R? 
cannot be an equilibrium of the LAGRANGE game A unless: 


(Ko) g(x) 2 0; 


(Ki) y7 g(x) = 0; 
(K3) VeL(x,y)d < 0 holds for alld such thatr+de xX. 


Proof. We already know that the feasibility condition (Ko) and the 
complementary slackness condition (K,) are necessarily satisfied by 
an equilibrium. If (K3) were violated and V,L(z, y)d > 0 were true, 
the z-player could improve the [-value by moving a bit into direction 


d. This would contradict the definition of an “equilibrium”. 


REMARK 3.4. The three conditions of Lemma 3.2 are the so-called 
KKT-conditions.. Although they are always necessary, they are not 


"C.G. Jacosr (1804-1851). 
SNamed after the mathematicians KARUSH, KUHN and TUCKER. 
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always sufficient to conclude that a candidate (x,y) is indeed an 
equilibrium. 


4.3. Shadow prices 


Given m functions aj,...,@m : Ri — R and m scalars }j,..., 
bm € R, the optimization problem 
max fle) Sit. aie) <Sbyj2 225 Oy Oe) Seg (26) 
xERT 


is of type (22) with the m restriction functions g;(x) = b; — a;(x) and 
has the LAGRANGE function 


L(x,y) = f(x) + > yi(bi — ai(z)) 
i=l 


= f(a) — S~ yai(x) + © yidi- 
i=l i=l 


For an intuitive interpretation of the problem (26), think of the data 
vector 


x = (£1,...,Ln) 


as a plan for n products to be manufactured in quantities x; and of 
f(x) as the market value of x. 

Assume that x requires the use of m materials in quantities 
ai(@),...,@m(x) and that the parameters b1,...,bm describe the 
quantities of the materials already in the possession of the manu- 
facturer. 

If the numbers y1,..., Ym represent the market prices (per unit) 
of the m materials, we find that L(ax,y) is the total value of the 
manufacturer’s assets: 


L(x,y) = market value of the production x 


+ value of the materials left in stock. 


The manufacturer would like to have that value as high as 
possible by deciding on an appropriate production plan zx. 
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“The market” is an opponent of the manufacturer and looks at 
the value 


~L(2,y) = S~ yilai(e) — bi) — fle), 
i=1 


which is the value of the materials the manufacturer must still buy on 
the market for the production of x minus the value of the production 
that the market would have to pay to the manufacturer for the 
production 2. 

The market would like to set the prices y; so that —L(x,y) is as 
large as possible. Hence: 


e The manufacturer and the market play a LAGRANGE 
game A. 


e An equilibrium (x*,y*) of A reflects an economic balance: 


Neither the manufacturer nor the market has a guaranteed 
way to improve their value by changing the production plan 
or by setting different prices. 


In this sense, the production plan 2* is optimal if (a*,y*) is an 
equilibrium. The optimal market prices yj,...,y;, are the so-called 
shadow prices of the m materials. 

The complementary slackness condition (K;) says that a material 
which is in stock but not completely used by x* has zero market value: 


Ge ob... Sy a0, 


The condition (K2) implies that x* is a production plan of optimal 
value f(z*) = L(a*,y*) under the given restrictions. Moreover, one 
has 


m m 
S- yfai(a*) = S- ytbi, 
1=1 i=1 


which says that the price of the materials used for the production «* 
equals the value of the inventory under the shadow prices y;. 
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Property (K3) says that the marginal change V,L(a2*,y*)d of 
the manufacturer’s value L is negative in any feasible production 
modification from x* to «* +d and only profitable for the market 
because 


V2(—L(2*, y*)) = —V2L(2*,y*). 


We will return to production games in the context of cooperative 
game theory in Section 8.3.2. 


4.4. Equilibria of convex LAGRANGE games 


Remarkably, the KKT-conditions turn out to be not only necessary 
but also sufficient for the characterization of equilibria in convex 
LAGRANGE games with differentiable objective functions. This gives 
a way to compute such equilibria and hence to solve optimization 
problems of type (22) in practice’: 


e Find a solution (x*,y*) € X x RY for the KKT- 
inequalities. (x*,y*) will yield an equilibrium in A = 


(X,R7,L) and x* will be an optimal solution for (22). 


Indeed, one finds: 


THEOREM 3.3. A pair (a*,y*) € X x RY} is an equilibrium 
of the convex LAGRANGE game A = (X,R,L) if and only if 


(a*,y*) satisfies the KKT-conditions. 


Proof. From Lemma 3.2, we know that the KKT-conditions are 
necessary. To show sufficiency, assume that (a*,y*) € X x R? 
satisfies the KKT-conditions. We must demonstrate that (x*,y*) is 


°It is not the current purpose to investigate further computational aspects 
in details, which can be found in the established literature on mathematical 
programming. 
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an equilibrium of the convex LAGRANGE game A = (X,R’?’, L) , i.e., 
satisfies 


max L(x, y") = L(x", y") min (x*,y) (27) 


for the function L(a, y) = f(x) + y7 g(a). 
Since « +» L(x,y*) is a concave function, we have for every 
reEX, 


L(z,y*) — L(a*,y*) < VaLb(2*,y*)( — 2°). 
Because (K2) guarantees V,L(a*, y*)(a — x*) < 0, we conclude 
L(x,y") < L(a*,y"), 


which implies the first equality in (27). For the second equality, (Ko) 
and (K,) yield g(x*) > 0 and (y*)"g(x*) = 0 and therefore: 


min L(x", y) = f(a") + min y! g(x") = f(#") +0 


= f(x") + (y*)" 9(2*) = La", y*). 


4.5. Linear programs 


A linear program (LP) in standard form is an optimization problem 
of the form 


max cla st. Ax <b, (28) 
reRt 


where c € R” and 6b € R™ are parameter vectors and A € R™*” 
a matrix, and thus is a mathematical optimization problem with a 
linear objective function f(x) = c’ 


b — Ax so that 


x and restriction function g(x) = 


gz) >0 «+> Axr<bd. 


The feasibility region F of (28) is the set of all nonnegative solutions 
of the linear inequality system Ax < b: 


F = P,(A,b) = {x € R” | Ax < b,x > O}. 
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The LAGRANGE function is 


L(a,y) = ca ty" (b— Ax) = y"b4+ (c? —y" A)z 


and yields for any x > 0 and y > 0: 


The optimum value of Lz is found by solving the dual associated 
linear program 


: : T T Te 
= t. ASG 2 
mu Da(y) = man Yb sas. yee (29) 
Ex. 3.6. Since y7b = b’y and (c’ —y? A)x = x7 (c— Ay) holds, the 
dual linear program (29) can be formulated equivalently in standard 
form: 


max (—b)’y st. (—A?)y <-e. (30) 
yeR? 


The main theorem on linear programming is: 


THOREM 3.4 (Main LP-Theorem). For the LP (28) the 
following holds: 


(A) An optimal solution x* exists if and only if both the LP 
(28) and the dual LP (29) have feasible solutions. 


(B) A feasible x* is an optimal solution if and only if there 
exists a dually feasible solution y* such that 


Cat = In(a*) = Loy =r yt. 


Proof. Assume that (28) has an optimal solution x* with value 
2* = clx*. Then c! x < 2z* holds for all feasible solutions x. So the 
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FARKAS Lemma!” guarantees the existence of some y* > 0 such that 
(gy AS and yb <2". 


Noticing that y* is dually feasible and that Li(x*) < Le(y) holds for 
all y > 0, we conclude that y* is, in fact, an optimal dual solution: 


Lo(y*) = (y*)b < 2 = Li(a*) < Laly*) = Ly (a*) = La(y*). 


This argument establishes property (B) and shows that the existence 
of an optimal solution necessitates the existence of a dually feasible 
solution. Assuming that (28) has at least one feasible solution z, it 
therefore remains to show that the existence a dual feasible solution 
y implies the existence of an optimal solution. 
To see this, note first 
w* = inf b’y > L(x) > —oo. 
y20 

So each dually feasible y satisfies —b’y < —w*. Applying now the 
FARKAS Lemma to the dual linear program in the form (30), we find 
that a parameter vector «* > 0 exists with the property 


Ac* <b and Iy(2*) > wu. 


* 


On the other hand, the primal-dual inequality yields Ly(a2*) < w*. 


So x«* must be an optimal feasible solution. 


General linear programs. In general, a linear program refers to the 
problem of optimizing a linear objective function over a polyhedron, 
namely the set of solutions of a finite system of linear equalities and 
inequalities and, therefore, can be formulated as 


max cla st. Ax <b,Br=d (31) 
xER” 


with coefficient vectors c € R", b € R™, d € R* and matrices A € 
IRs" and BS Res, 


See Lemma A.6 in the Appendix. 
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If no equalities occur in the formulation (31), one has as a linear 
program in canonical form: 


max ca s.t. Ax <b. (32) 
xER” 


Because of the equivalence 
Bra=d <= Ba<dand—-B< -—d, 


the optimization problem (31) can be presented in canonical form: 


A b 
max cx st. Bia<t|d 
E n 
- =B —d 
Moreover, since any vector « € R” can be expressed as the 
difference 
g=at —a7 


of two (nonnegative) vectors «*,x~ € Rt, one sees that each linear 
program in canonical form is equivalent to a linear program in 
standard form: 
max cha —chx st. Axt — Ax <b. 
xt,e2->0 

The LAGRANGE function of the canonical form is the same as 
for the standard form. Since the domain of Ly; is now X = R”, the 
utility function L2(y) differs accordingly: 


if Ar <b 
if Ax < b. 


ify A=e 
eye eae 


Relative to the canonical form, the optimum value of Lz is found 
by solving the linear program 


in Lo(y)=min yb st. ylA=c!. 
min 2(y) Ble ash: sat SY c (33) 
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Nevertheless, it is straightforward to check that Theorem 3.4 is 
literally valid also for a linear program in its canonical form. 

Linear programming problems are particularly important in 
applications because they can be solved efficiently. In the theory 
of cooperative games with possibly more than two players (see 
Chapter 8), linear programming is a structurally analytical tool. We 
do not go into algorithmic details here but refer to the standard 


mathematical optimization literature.!! 


4.6. Linear programming games 
A linear programming game is a LAGRANGE game that arises from 
a linear program. If c € R”,b € R™ and A € R™*” are the problem 
parameters, we may denote is by 

I4(¢, A, b) = A(R4, R”, L) 

L(c, A, b) = A(R", R™, L), 

where 

L(z,y) =cla +y™b- y" Ax 


is the underlying LAGRANGE function. Linear programming games 
are convex. In particular, Theorem 3.4 implies: 


(1) A linear programming game admits an equilibrium if and 
only if the underlying linear program has an optimal 
solution. 


(2) The equilibria of a linear programming game are precisely 


the pairs (2*, y*) of primally and dually optimal solutions. 


Ex. 3.7. Show that every randomized matrix game is a linear 
programming game. (See Section 3.3.2.) 


‘4 ¢.9., FAIGLE et al. [17]. 
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Chapter 4 


Investing and Betting 


The opponent of a gambler is usually a player with no specific optimization 
goal. The opponent’s strategy choices seem to be determined by chance. 
Therefore, the gambler will have to decide on strategies with good 


expected returns. Information plays an important role in the quest for 
the best decision. Hence the problem how to model information exchange 
and common knowledge among (possibly more than two) players deserves 
to be addressed as well. 


Assume that an investor (or bettor or gambler or simply player) is 
considering a financial engagement in a certain venture. Then the 
obvious — albeit rather vague — big question for the investor is: 


e What decision should best be taken? 


More specifically, the investor wants to decide whether an 
engagement is worthwhile at all and, if so, how much of the available 
capital should be invested how. Obviously, the answer depends on 
additional information: What is the likelihood of a success? What 
gain can be expected? What is the risk of a loss? etc. 

The investor is thus about to participate as a player in a 2-person 
game with an opponent whose strategies and objective are not always 
clear or known in advance. Relevant information is not completely 
(or not reliably) available to the investor so that the decision must 
be made under uncertainties. Typical examples are gambling and 
betting where the success of the engagement depends on events that 
may or may not occur and hence on “fortune” or “chance”. But also 


75 
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investments in the stock market fall into this category when it is not 
clear in advance whether the value of a particular investment will 
rise or fall. 

We are not able to answer the big question above completely but 
will discuss various aspects of it. Before going into further details, 
let us illustrate the difficulties of the subject with a classical — and 
seemingly paradoxical — gambling situation. 


The St. Petersburg paradox. Imagine yourself as a potential 
player in the following game of chance. 


Ex. 4.1 (St. Petersburg game). A coin (with faces “H” 
and “T”) is tossed repeatedly until “H” shows. If this happens 
at the nth toss, a participating player will receive a, = 2” 
euros. There is a participation fee of ag euros, however. So the 
net gain of the player is 


@ = An — a9 = 2” — ag 


if the game stops at the nth toss. At what entrance fee ag 


would a participation in the game be attractive? 


Assuming a fair coin in the St. Petersburg game, the probability 
to go through more than n tosses (and hence to have the first n 
results as “T”) is 


m=($) =5 > 0 (noo). 


So the game ends almost certainly after a finite number of tosses. 
The expected return to a participant is nevertheless infinite: 


oO 91 92 n 
= n = | | = ff 
eS ie sg agg Tg hh OP 


which might suggest that a player should be willing to pay any finite 
amount ao for being allowed into the game. In practice, however, this 
could be a risky venture (see Ex. 4.2). 
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Ex. 4.2. Show that the probability of receiving a return of 100 euros 
or more in the St. Petersburg game is less than 1%. So a participation 
fee of ag = 100 euros or more appears to be not attractive because 
it will not be recovered with a probability of more than 99%. 


Paradoxically, when we evaluate the utility of the return a, = 2” 
not directly but by its logarithm logga, = log,2” = n, the St. 
Petersburg payoff has a finite utility expectation: 


logs 2' — logy 2? logy 2” aa | 
Gp= nar ee Lo 


This observation suggests that one should expect an utility value of 
less than 2 and hence a return of less than 2? = 4 euros. 


REMARK 4.1 (Logarithmic utilities). The logarithm function as 
a measure for the utility value of a financial gain was introduced by 
D. BERNOULLI! in his analysis of the St. Petersburg game. This 
concave function plays an important role in our analysis as well. 

Whether one uses log, x, the logarithm base 2, or the natural 
logarithm Inz does not make an essential difference since the two 
functions differ just by a scaling factor: 


lng (12): loge.a. 


Arithmetic and geometric growth. If investments are repeated, 
the question arises how to evaluate the evolution of the capital arising 
from the investments. To make this question more precise, assume 
that a capital of initial size bp takes on the values 


bo, by,.--, 045... 
in discrete time steps. The arithmetic growth rate up to time t is the 
number 
b, — 6 
— — and hence bh; = bp + ta. 


'D. BERNOULLI (1700-1782). 
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The geometric growth rate up to time t is the number 


vy = %/t:/bo and hence b; = 7f - bo. 


Passing to logarithms, the geometric growth rate becomes the 
logarithmic geometric growth rate 


In b, — ln bo 


= (34) 


G,=ny% = 
which is the arithmetic growth rate of the logarithmic utility. Since 
x ++ Inz is a monotonically strictly increasing function, optimizing 
yz is equivalent with optimizing of G;. In other words: 


Optimizing the geometric growth rate is equivalent with 
optimizing the arithmetic growth rate of the logarithmic 


utility. 


1. Proportional investing 


Our general model consists of a potential investor with an initial 
portfolio B of 6 > 0 euros (or dollars or...) and an investment 
opportunity A. If things go well, an investment of size x would bring 
a return rx > x. If things do not go well, the investment will return 
nothing. 


In the analysis, we will denote the net return rate by 
p=r-1. 


The investor is to decide what portion of B should be invested. 
The investor believes: 


(PI) Things go well with probability p > 0 and do not go well with 
probability gq = 1 — p. 
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1.1. Eapected portfolio 


Under the assumption (PI), the investor’s expected portfolio value 
after the investment 2 is 


B(x) = [(b—@) + ralp + (6— x)q = [b+ palp + (b— x)q 


since an amount of size b — x is not invested and therefore not at 
risk. The derivative is 


B'(x) = pp—4q 


So B(x) is strictly increasing if p > q/p and non-increasing otherwise. 
Hence, if the investor’s decision is motivated by the maximization of 
the expected portfolio value B(x), the naive investment rule applies: 


(NIR) If p > q/p, invest all of B in A and expect the return 
B(b) = rbp = (1+ q)b > b. 


If p < q/p, invest nothing since no proper gain is 
expected. 


In spite of its intuitive appeal, rule (NIR) can be quite risky (see 
Ex. 4.3). 


Ex. 4.3. Assume that the investment opportunity A offers the 
return rate r = 100 for an investment in the case it goes well. Assume 
further that an investor estimates the probability for A to go well to 
be p = 10%. In view of 


q/p=9 < 99=r-l=p, 


a naive investor would invest the full amount «* = 6 and expect a 
tenfold return: 


B(a*) = B(b) = rbp = 100 


However, with probability q = 90%, the investment should be 
expected to result in a total loss. 
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1.2. Expected utility 


With respect to the logarithmic utility function Inz, the expected 
utility of an investment of size x with net return rate p would be 


U(x) = pln(b+ px)) + qln(b— 2). 


The marginal utility of x is the value of the derivative 


PP qd 


/ — 
eZ eames b-x 


The second derivative is 


So the investment x* has optimal (logarithmic) utility if 


U' (e*) =0: “or = 

Ex. 4.4. In the situation of Ex. 4.3, one has 
iC en Cee Bea ee 
t)= 75 ln ) + 75 in xe 


with the derivative 


a: 9 
U'®) = 1964992) — iba)" 


(35) 


U'(x*) = 0 implies x* = b/11. Hence the portion a* = 1/11 of the 
portfolio should be invested in order to maximize the expected utility 
U. The rest of the portfolio should be retained and not be invested. 

The expected value of the portfolio with respect to x* = b/11 is 


B(a*) = B(b/11) = (1+ 9/11) © 1.818 b 


and thus much lower than under the naive investment policy. 
However, the investor is guaranteed to preserve at least 10/11 of 


the original portfolio value. 
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1.3. The fortune formula 


Let a = x/b denote the fraction of the portfolio B to be possibly 
invested into the opportunity A with r-fold return if A works out. 
With p = r—1 > 0, the expected logarithmic utility function then 
becomes 


u(a) = U(a/b) = pInb(1 + pa/b) + qin b(1 — x/b) 


with the derivative 


(36) 


If a loss is to be expected with positive probability gq > 0, and 
the investor decides on a full investment, 7.e., chooses a = 1, then 
the expected utility (value) 


Uh) = lim u(a) = —oo 


results — no matter how big the net return rate p might be. 
On the other hand, the choice a = 0 of no investment has the 
utility 


u(0) = Inb. 


The investment fraction a* yielding the optimal utility lies some- 
where between these extremes. 


LEMMA 4.1 Let u’(a) be as in (36) and 0 < a* <1. Then 


u'(a*) =0 a*=p-—4q/p. 


Proof. Exercise left to the reader. 


The choice of the investment rate a* with optimal expected 
logarithmic utility u(a*) yields the so-called fortune formula of 
KELLY [24]: 
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Cpe Ore eal 
p p 


An investment strategy according to (37) is known as a G-strategy 
or KELLY strategy.” 


Betting one’s belief. It is important to keep in mind that the 
probability p in the fortune formula (37) is the subjective evaluation 
of an investment success by the investor. 

The “true” probability is often unknown at the time of the 
investment. However, if p reflects the investor’s best knowledge about 
the true probability, there is nothing better the investor could do. 
This truism is known as the investment advice 


Bet your belief! 


2. Fair odds 


An investment into an opportunity A offering a return of r > 1 
euros per euro invested with a certain probability Pr(A) or returning 
nothing (with probability 1—Pr(A)) is called a bet on A. The investor 
is then a bettor (or a gambler) and the net return 


p=r-1l 


is the payoff of the bet. The payoff is assumed to be guaranteed by 
a bookmaker (or bank). The net return rate is also denoted® by p: 1 
and known as the odds of the bet. 

The expected gain (per euro) of the gambler, and hence the 
bookmaker’s loss, is 


E = pPr(A) + (—1)(1 — Pr(A)) =r Pr(A) —- 1. 


See, e.g., ROTANDO AND THORP [38]. 
3The notation differs in various parts of the world: in continental Europe, for 
example, the gross return rate r: 1 is customary. 
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The odds p : 1 are considered to be fair if the gambler and the 
bookmaker have the same expected gain, 7.e., if 


E=-E andhence L=0 


holds. In other words: 


p:1- is fair 


If the true probability Pr(A) is not known to the bettor, it needs 
to be estimated. Suppose the bettor’s estimate for Pr(A) is p. Then 
the bet appears (subjectively) advantageous if and only if 


E(p)>0, te. ifr >1/p. (38) 
The bettor will consider the odds p: 1 as fair if 
E(p) =0 and hence r = 1/p. 


In the case E(p) < 0, of course, the bettor would not expect a gain 
but a loss on the bet — on the basis of the information that has led 
to the subjective probability estimate p for Pr(A). 


Examples. Let us look at some examples. 


Ex. 4.5 (DE MERE’s game‘). Let A be the event that no “6” 
shows if a single 6-sided die is rolled four times. Suppose the odds 
1:1 are offered on A. If the gambler considers all results as equally 
likely, the gambler’s estimate of the probability for A is 


54 625 


— = —— *® 0.482 : 
Gi ~ [06 0.482 < 0.5 


p= 


because there are 64 = 1296 possible result sequences on 4 rolls of 
the die, of which 54 = 625 correspond to A. So the player should 


“Mentioned to B. PASCAL (1623-1662). 
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expect a negative return: 
E(p) = (p+ 1)p—1= 2p —1=2(5/6)4 -1 <0. 


In contrast, let A be the event that no double 6 shows if a pair of 


dice is rolled 24 times. Now the prospective gambler estimates Pr(A) 
as 


p = (35/36)4 > 0.5. 


Consequently, the odds 1: 1 on A would lead the gambler to expect 
a proper gain: 


E=2%-1>0. 


Ex. 4.6 (Roulette). Let W = {0,1,2,...,36} represent a roulette 
wheel and assume that 0 € W is colored green while eighteen 
numbers in W are red and the remaining eighteen numbers black. 
Assume that a number X € W is randomly determined by spinning 
the wheel and allowing a ball come to rest at one of these numbers. 


(a) Fix w € W and the gross return rate r = 18 on the event 
ApHt{A =u}, 


Should a gambler expect a positive gain when placing a bet 
on Ay? 

(b) Suppose the bank offers the odds 1: 1 on the event R = {X = 
red}. Should a gambler consider these odds on R to be fair? 


The doubling strategy. For the game of roulette (see Ex. 4.6) and 
for similar betting games with odds 1: 1 (i.e., with the return rate 
r = 2) a popular wisdom? recommends repeated betting according 
to the following strategy: 


°T have learned strategy (D) myself as a youth from my uncle Max. 


2. Fair odds 85 


(D) Bet the amount 1 on R= {X = red}. If R does not occur, 
continue with the double amount 2 on R. If R does not 


show, double again and bet 4 on R and so on — until the 
event R happens. 


Once R shows, one has a net gain of 1 on the original investment 
of size 1 (see Ex. 4.7). The probability for R not to happen in one 
spin is 19/37. So the probability of seeing red in one of the first n 
spins of an equally balanced roulette wheel is high: 


1 — (19/37)” > 1 (now). 


Hence: Strategy (D) achieves a net gain of 1 with high probability. 
Paradoxically(?), the expected net gain for betting any amount 
x > 0 on the event RF is always strictly negative however: 


x 


[en eee = < 0 
ace 01s MEY 


Ex. 4.7. Show for the game of roulette with a well-balanced wheel: 


(1) If {X = red} shows on the fifth spin of the wheel only, strategy 
(D) has lost a total of 15 on the first 4 spins. However, having 
invested 16 more and then winning 2° = 32 on the fifth spin, 
yields the overall net return 


32 — (15 +16) =1. 


(2) The probability for {X = red} to happen on the first 5 spins is 
more than 95%. 


REMARK 4.2. The problem with strategy (D) is its risk manage- 
ment. A player has only a limited amount of money available in 
practice. If the player wants to limit the risk of a loss to B euros, 
then the number of iterations in the betting sequence is limited to 
at most k, where 


ak-1<B<2* andhence k= |log, BI. 
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Consequently: 


e The available budget B is lost with probability (19/37)*. 
e The portfolio grows to B+ 1 with probability 1 — (19/37)*. 


3. Betting on alternatives 


Consider k mutually exclusive events Ao, Ai1,..., Ax—1 of which one 
will occur with certainty and a bank that offers the odds p; : 1 for 
bets on the & events A;, which means: 


(1) The bank guarantees a total payoff of r; = p; +1 euros for 
each euro invested in A; if the event A; occurs. 


(2) The bank offers a scenario with 1/r; being the probability 
for A; to occur. 


Suppose a gambler estimates the events A; to occur with 
probabilities p; and decides to invest the capital B of unit size® 
b = 1 fully. Under this condition, a (betting) strategy is a k-tuple 
a = (do, @1,-..,@,—1) of numbers a; > 0 such that 


ag tayt+...+ap_1 = 1 


with the interpretation that the portion a; of the capital will be bet 
onto the occurrence of event A; fori = 0,1,...,4—1. The gambler’s 
expected logarithmic utility of strategy a is 


k-1 k-1 
= pina + >— pjln7y. 
i=0 i=0 
Notice that p = (po, p1,---,Pr—1) is a strategy in its own right 
and that the second sum term in the expression for U(a,p) does not 


®The normalized assumption results in no loss of generality: the analysis is 
independent of the size of B. 
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depend on the choice of a. So only the first sum term is of interest 
when the gambler seeks a strategy with optimal expected utility. 


THEOREM 4.1. Let p = (po,pi,---,;Pp—1) be the gambler’s 
probability assessment. Then: 


U(a,p)<U(p,p) = > aFfp. 


Consequently, a* = p is the strategy with the optimal logarith- 
mic utility under the gambler’s expectations. 


Proof. The function f(x) = x —1-—lInz is defined for all x > 0. Its 
derivative 


fie) =1- 1a 


is negative for x < 1 and positive for x > 1. So f(x) is strictly 
decreasing for x < 1 and strictly increasing for x > 1 with the unique 
minimum f(1) = 0. This yields BERNOULLI’s inequality 


lig <a =-L and dng =e) aS aS 1 (39) 


Applying the BERNOULLI inequality, we find 


k-1 
U(a, Pp) -¥n hia. = Ye Inp; = Soi In(a;/pi) 
1=0 
< DG pea) -Sa- ae = 0 


with equality if and only if a; = p; for alli =0,1,..., hy. 


Theorem 4.1 leads to the betting rule with the optimal expected 
logarithmic utility: 


(BR) For alli = 0,1,...,k —1, bet the portion a; = p; of the 


capital B on the event Aj. 
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REMARK 4.3. The proportional rule (BR) depends only on the 
gambler’s probability estimate p. It is independent of the particular 
odds p; : 1 the bank may offer! 


Fair odds. As in the proof of Theorem 4.1, one sees: 


k-1 k-1 k-1 k-1 
So pilnr; = — >> pj n(Q1/ri) > — So pimp; = Spi n(1/pi) 
i=0 i=0 i=0 i=0 


with equality if and only if r; = 1/p; holds for alli = 0,1,...,k—1. 
It follows that the best odds p; : 1 for the bank (and worst for the 
gambler) are given when 


1- 4 
Cael ae GS 6f eed): (40) 
4 


In this case, the gambler expects the logarithmic utility of the 
optimal strategy p as 


k-1 k-1 
U(p) = So pilnpi — > pi n(1/rs) = 0. 
i=0 i=0 


We understand the odds as in (40) to be fair in the context of 
betting with alternatives. 


3.1. Statistical frequencies 


Assume that the gambler of the previous sections has observed: 


e The event A; has popped up s; times in n consecutive instances of 
the bet. 


So, under the strategy a, the original portfolio B of unit size 
b = 1 would have developed into size 


B,(a) = (aoro)*? (a1r1)* «+» (@g—17rR—1) "#3 
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with the logarithmic utility 


k-1 


U (a= in Bp (a),= S- s; In(a;r;) 


As in the proof of Theorem 4.1, the gambler finds in hindsight: 


COROLLARY 4.1. The strategy a* = (s0/n,...,5%~1/n) 
would have led to the maximal logarithmic utility value 


k-1 k-1 
Un(a*) = 5_ si In(si/t) + 5° pln 
1=0 i=0 


and hence to the growth with the maximal geometric growth 
rate 
(Gong) AST eee Sreinne ace 

nn ; 


By tae 


Based on the observed frequencies s; the gambler might reason- 
ably estimate the events A; to occur with probabilities according to 
the relative frequencies 


R=sin- C=O). 255k —1) 


and then expect optimal geometric growth. 


4. Betting and information 


Assuming a betting situation with the k& alternatives Ap, Aj,..., 
Ax_; and the odds p, : 1 (for x = 0,1,...,k —1) as before, suppose, 
however, that the event A, is actually already established — but that 
the bettor does not have this information before placing the bet. 
Suppose further that information now arrives through some 
(human or technical) communication channel K so that the outcome 
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A, is reported to the bettor (perhaps incorrectly) as Ay: 


bby 99 


e Having received the (“insider”) information “y”, how 
should the bettor place the bet? 


The question is: 


To answer this question, let 
p(x|y) = probability for the true result to be x when y is received. 


REMARK 4.4. The parameters p(|y) are typically subjective eval- 
uations of the bettor’s trust in the channel K. 


A betting strategy in this setting of information transmission 
becomes a (k x k)-matrix A with coefficients a(a|y) > 0 which satisfy 


k-1 

S| a(xly) =1° for y= 0,1)...¢k— 1, 

x=0 
According to strategy A, a(x|y) would be the fraction of the budget 
that is bet on the event A, when y is received. In particular, the 
bettor’s trust matrix P with the coefficients p(aly) is a strategy. 
In the case where A, is the true result, one therefore expects the 
logarithmic utility 


k-1 k-1 
Ur(A) = S° p(aly) nfa(aly)re] = S© p(aly) na(ely) + Inre. 
y=0 y=0 


As in Corollary 4.1, we find for all  =0,1,...,4 —1: 
UA) 2 0GP) eS (ale) Hoey) Vee Ol ee 


So the trust matrix P yields also an optimal strategy (under the 
given trust in K on part of the bettor) and confirms the betting rule: 


Bet your belief! 
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Information transmission. Let p = (2%0,%1,...,Up-1) be the 
bettor’s probability estimates on the k events Ap, Aj,...,A,z—1 OF, 
equivalently, on the index set {0,1,...,4 — 1}. Then the expected 
logarithmic utility of strategy A is relative to base 2: 


k-1k-1 


US? (A) = S~S~ pep(aly) logs a(ely) “Sn. logs Te 
x=0 y=0 


Nota BENE. The probabilities pz are estimates on the likelihood of 
the events A,, while the probabilities p(x|y) are estimates on the 
trust into the reliability of the communication channel K. They are 
logically not related. 

Setting 


k-1 
== So De logs t 
i=1 


k-1 


Ht) = —S re logy x 


i=l 
k-1k-1 
= Ss" S > pep(aly) logs a(x|y), 
x=0 y=0 
we thus have 
Us?) = —H(X|Y) - H(r) = Uo(p) + T(XIY) 
where 


T(X|Y) = H(X) — H(XIY) 


is the increase of the bettor’s expected logarithmic utility due to the 
communication via channel K. 


REMARK 4.5 (Channel capacity). Given the channel K as above 
with transmission (trust) probabilities p(s|r) and the probability 
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distribution 


p = (pz|t € X) 


on the channel inputs x, the parameter T(X,Y) is the (information) 
transmission rate of K. 

Maximizing over all possible input distributions p, one obtains 
the channel capacity C(i) as the smallest upper bound on the 
achievable transmission rates: 


C(K) =sup T(X,Y). 


The parameter C(K) plays an important role in the theory of 
information and communication in general.’ 


Ex. 4.8. A bettor expects the event Ag with probability 80% and 
the alternative event A, with probability 20%. What bet should be 
placed? 

Suppose now that an expert tells the bettor that A, is certain 
to happen. What bet should the bettor place under the assumption 
that the expert is right with probability 90%? 


5. Common knowledge 


Having discussed information with respect to betting, let us digress 
a little and take a more general view on information and knowledge.® 
Given a system G, we ask: 


To what extent does common knowledge in a group of agents 


influence individual conclusions about the state of G? 


To explain more concretely what is meant here, we first discuss 
a well-known riddle. 


"See SHANNON [42]. 
8More can be found in, e.g., FAGIN et al. [10]. 
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5.1. Red and white hats 


Imagine the following situation: 


(1) Three girls, Gj, Gz and G3, with red hats sit in a circle. 
(II) Each girl knows that their hats are either red or white. 
(III) Each girl can see the color of all hats except her own. 


Now the teacher comes and announces: 


(1) There is at least one red hat. 
(2) I will start counting slowly. As soon as someone knows the 
color of her hat, she should raise her hand. 


What will happen? Does the teacher provide information that 
goes beyond the common knowledge the girls already have? After 
all, each girl sees two red hats — and hence knows that each of the 
other girls sees at least one red had as well. 

Because of (III), the girls know their hat universe 9 is in one of 
the 8 states of possible color distributions: 


None of these states can be jointly ruled out. The entropy H9 of their 
common knowledge is: 


H3 = log, 8 = 3. 


The teacher’s announcement, however, rules out the state og and 
reduces the entropy to 


Hi = logs 7 < HS, 


which means that the teacher has supplied proper additional infor- 
mation. 

At the teacher’s first count, no girl can be sure about her own 
hat because none sees two white hats. So no hand is raised, which 
rules out the states 05,06 and o7 as possibilities. 
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Denote now by P;(c) the set of states thought possible by girl G; 
when the hat distribution is actually a. So we have, for example, 


P, (3) = {a3}, Po(o2) = {o2}, P3(o4) = {oa}. 


Consequently, in each of the states o2,03,04, at least one girl would 
raise her hand at the second count and conclude confidently that 
her hat is red, which would signal the state (and hence the hat 
distribution) to the other girls. 

If no hand goes up at the second count, all girls know that they 
are in state o, and will raise their hands at the third count. 

In contrast, consider the other extreme scenario and assume: 


(’) Three girls, Gj, Gz and G3, with white hats sit in a circle. 
(II) Each girl knows that their hats are either red or white. 
(III) Each girl can see the color of all hats except her own. 


The effect of the teacher’s announcement is quite different: 


e Each girl will immediately conclude that her hat is red and raise 
her hand because she sees only white hats on the other girls. 


This analysis shows: 


(i) The information supplied by the teacher is subjective: Even 
when the information (“there is at least one red hat”) is 
false, the girls will eventually conclude with confidence that 


they know their hat’s color. 
(ii) When a girl thinks she knows her hat’s color, she may never- 
theless have arrived at a factually wrong conclusion. 


Ex. 4.9. Assume an arbitrary distribution of red and white hats 
among the three girls. Will the teacher’s announcement nevertheless 
lead the girls to the belief that they know the color of their hats? 
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5.2. Information and knowledge functions 


An event in the system G is a subset E£ C G of states. We say that 
the event E occurs when G is in a state o € E. Denoting by 2© the 
collection of all possible events, we think of a function P : G6 + 2° 
with the property 


0€P(o) foralaeG 
as an information function. P has the interpretation: 


e If G is in the state o, then P provides the information that the 
event P(o) has occurred. 


Notice that P is not necessarily a sharp identifier of the “true” 
state o: any state r € P(c) is a candidate for the true state under 
the information function P. 

The information function P defines a knowledge function K : 
25 + 2° via 


K(E) = {a | P(o) C E} 
with the interpretation: 


e K(E) is the set of states o € G where P suggests that the event E 
has certainly occurred. 


LEMMA 4.2. The knowledge function K of an information 
function P has the properties: 


(K.1) K(6) =6. 

BeP = Re, 
(ENF) =K(E)NK(F). 
(ies, 


(K.2) 
(K.3) 
(K.4) 


Kk 
Kk 


Proof. Straightforward exercise, left to the reader. 


Property (K.4) is the so-called reliability axiom: If one knows 
(under A’) that FE has occurred, then E really has occurred. 
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Ex. 4.10 (Transparency). Verify the transparency axiom 
(K.5) K(K(E£)) = K(E) for all events E. 


Interpretation: When one knows with certainty that E has 
occurred, then one knows with certainty that one considers E 
as having occurred. 


We say that E is evident if E = K(E) is true, which means: 


e The knowledge function K considers an evident event FE as having 
occurred if and only if & really has occurred. 


Ex. 4.11. Show: The set G of all possible states constitutes always 
an evident event. 


Ex. 4.12 (Wisdom). Verify the wisdom axiom 
(K.6) G \ K(£) = K(G \ E) for all events EL. 


Interpretation: When one does not know with certainty that E 
has occurred, then one is aware of one’s uncertainty. 


5.3. Common knowledge 


Consider now a set N = {pj,..., pn} of n players p; with respective 
information functions P; and knowledge functions K;. We say that 
the event EF C G is evident for N if E is evident for each of the 
members of JN, i.e., if 


E=K,(E)=...=K,(E). 


More generally, an event EF C G is said to be common knowledge 
of N in the state o if there is an event F C EF such that 


F is evident for N and o EF. 
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PROPOSITION 4.1. If the event E C G is common knowledge 
for the n players p; with information functions P,; in state o, 
then 


PUR (EO en (Kee) 


holds for all sequences i, ...im of indices 1 <1; <n. 


Proof. Ifthe event E is common knowledge, it comprises an evident 
event F C EF with o € F.. By definition, we have 


SRK CK.) SF 


By property (K.2) of a knowledge function (Lemma 4.2), we thus 
conclude 


Hohe (St Re Cn) 2 SR 


tm 


Pnweree. 


As an illustration of Proposition 4.1, consider the events 
Ky (E), K2(Ki(E)), K3(Ko (Ai (F))). 


K(£) are all the states where player p; is sure that E has occurred. 
The set K(k 1(E)) comprises those states where player p2 is sure 
that player p;(E) is sure that FE has occurred. In K3(Ko(h1(£))) 
are all the states where player pg is certain that player pg is sure that 
player p; believes that F has occurred. And so on. 


5.4. Different opinions 
Let p; and pg be two players with information functions P, and P» 


relative to a finite system G and assume: 


e Both players have the same probability estimates Pr(£) on the 
occurrence of events # C G. 
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We turn to the question: 


e Can there be common knowledge among the two players in a 
certain state o* that they have different likelihood estimates ny, 
and n2 for an event E having occurred? 


Surprisingly(?), the answer can be “yes” as Ex. 4.13 shows. For 
the analysis in the example, recall that the conditional probability of 
an event E given the event A, is 

Pr(£O A)/Pr(A) if Pr(A) > 
papa = JPEMA)/PH(A) if Pr(A) 
0 if Pe A):= 


Ex. 4.13. Let G = {01,02} and assume Pr(o,) = Pr(a2) 
1/2. Consider the information functions 


Pi(o1) ={o1} and Pi(o02) = {a2} 
P2(01) = {01,02} = Po(02). 
For the event F = {0}, one finds 
Pr(E|Pi(o1)) =1 and Pr(E|Pi(o2)) =0 
Pr(E|P2(01)) =1/2 and Pr(E|P2(o2)) = 1/2. 


The ground set G = {01,02} corresponds to the event “the 
two players differ in their estimates on the likelihood that E 
has occurred”. © is (trivially) common knowledge in each of 


the two states 01,09. 


For a large class of information functions, however, our initial 
question has the guaranteed answer “no”. For an example, let us call 
an information function P strict information function if 


(St) Every evident event E is a union of pairwise disjoint sets 
P(o). 
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PROPOSITION 4.2. Assume that both information functions 
P, and P, are strict. Let E C G be an arbitrary event. Assume 
that it is common knowledge for the players p, and p, in state 
o* €G that they estimate the likelihood for E having occurred 


as Hy, resp. N2. Then necessarily equality n, = no holds. 
Hence, if yn. # 2, the players’ likelihood estimates cannot be 
common knowledge. 


Proof. Consider the two events 
Ey = {a | Pr(E|Pi(c)) =m} 
Ey = {a | Pr(E|Po(o)) = np}. 


Exactly in the event FE, 9 E2, player p; estimates the probability 
for the event E having occurred to be 7 while player po’s estimate 
is 12. 

Suppose £, % Eg is common knowledge in state o*, i.e., there 
exists an event F’ C #, 9 E> such that 


o€F and Ki (F) =F= Ko(F). 


Because the information function P; is strict, F is the union of 
pairwise disjoint sets Pj(o1),..., Pi (a), say. Because of F C FE, Eo, 
one has 


Pr(E|Pi(o1)) = +++ = Pr(E|Pi(ox)) = m- 
Taking Ex. 4.14 into account, we therefore find 


Pr(E|F) = Pr(E|Pi(oi) = m. 


Similarly, Pr(E|F’) = nz is deduced and hence 72 = 7; follows. 


Ex. 4.14. Let A,B be events such that AN B = @. Then the 
conditional probability satisfies: 


Pr(E|A) = Pr(E|B) = Pr(E|AU B) = Pr(BA). 
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Part 3 


n-Person Games 
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Chapter 5 


Potentials, Utilities and Equilibria 


Before discussing n-person games per se, it is useful to go back to the 
fundamental model of a game I being played on a system G of states and 


look at characteristic features of [. The aim is a general perspective on 
the numerical assessment of the value of states and strategic decisions. 


1. Potentials and utilities 
1.1. Potentials 


To have a “potential” means to have the capability to enact 
something. In physics, the term potential refers to a characteristic 
quantity of a system whose change results in a dynamic behavior 
of the system. Potential energy, for example, may allow a mass to 
be set into motion. The resulting kinetic energy corresponds to the 
change in the potential. Gravity is thought to result from changes 
in a corresponding potential, the so-called gravitational field, and so 
on. 

Mathematically, a potential is represented as a real-valued 
numerical parameter. In other words: A potential on the system G 
is a function 


v:675R 


which assigns to a state 0 € G a numerical value v(c). Of interest is 
the change in the potential resulting from a state transition 0 > T: 


Ov(a,T) = v(T) — v(o) 


103 


104 5. Potentials, Utilities and Equilibria 


In fact, up to a constant, the potential v : G > R is determined by 
its marginal potential dv: G x 6 > R: 


LEMMA 5.1 For any potentials v,w : G6 — R, the two 
statements are equivalent: 


(1) dv =0w. 


(2) There exists a constant Ko € R such that for allo € 6, 


w(o) =v(0) + Ko. 


Proof. In the case (2), one has 
O(0,T) = v(T) — v(o) = w(T) — w(c) = Ow(o,T) 


and therefore (1). Conversely, if (1) holds, choose any og and set 
Ko = w(00) — v(o0). Then for all o € G, property (2) is apparent: 


w(a) = w(o0) + Ow(o0, 0) 


= Ko + v(90) + Ov(90,0) = Ko + v(o). 


1.2. Utilities 


A utility measure is a function 
U:6xG6G—-R, 


which we may interpret as a measuring device for the evaluation of 
a system transition 0 > 7 by a real number U(o,T). With U, we 
associate the so-called local utility functions u° : G — R with the 
values 


u’(t) =U(o,T) (0,7 €G). 


Hence the utility measure U can be equally well understood as an 
ensemble of local functions u? : G > R: 


UeRo = 455, iF eR ees 
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Path independence. Given the utility measure U, a path 


y= (09 9 01 9 02 >... 9 OR-1 > OR) 
of system transitions has the aggregated utility weight 
U(y) = U(o0, 01) + U(o1, 02)... + U(n-1, OK): 


We say that U is path independent if the utility weight of any path 
depends only on its initial state og and the final state ox: 


U(o0 OU ramet OF Yoteer are > On) = U(o0, 0K). 


Ex. 5.1. Show that U € R©*® is path independent if and only if 
the aggregated value of every circuit (1.e., path that starts and ends 
in the same state) is zero: 


U(o9 9 01 9... 9 OK 4 90) = O. 


In particular, a path-independent U satisfies U(o,a) = 0. 


PROPOSITION 5.1. The utility measure U € R©*© is path 


independent if and only if U = Ou holds for some potential 
ens 


Proof. If U is derived from the potential u: G — R, we have 


U (G;-45,.05) = Ou(o;, 01-1) = u(o;) = u(oj-1) 


and, therefore, for any y = (09 > 01 >... > Op-1 > Ox): 
k-1 
U(y) = So uli) — So u(oi) = u(ox) — uloo) = U(op, 00), 
i=1 i=0 


which shows that U is path independent. 
Conversely, assume that U is a path independent utility measure. 
Fix a state oo and consider the potential u € R© with values 


u(o) = U(o0, 0). 
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Since U is path independent, we have for all o,7 € G, 
U(o0,0) + U(o,00) =0 and hence U(o,09) = —U(o9,¢), 


which implies 


U(o,7T) = U(o, 00) + U(o0, T) = —u(o) + u(7) = Ou(o,7). 


Utility potentials. We refer to v € R®© as the underlying utility 
potential of a context that uses the marginal potential Ov as its utility 
measure. Such utility measures are particularly important in appli- 
cation models. The utility measures of cooperative game theory,! 
for example, are typically derived from potentials. 


2. Equilibria 


When we talk about an “equilibrium” of an utility measure U € 


RS&*S on the system G6, we make the prior assumption that each 


state o has associated a neighborhood 
F° CG withoe fF’ 


and that we concentrate on state transitions to neighbors, i.e., to 
transitions of type 0 > 7 with 7 € F”. 

We now say that a system state 0 € G is a gain equilibrium of 
U if no feasible transition o + 7 to a neighbor state 7 has a positive 
utility, z.e., if 


U(o,T) <0 holds for all tT € F. 
Similarly, o is a cost equilibrium if 
U(o,T) >0_ holds for all t € F°. 


REMARK 5.1 (Gains and costs). The negative C = —U of the 
utility measure U is also a utility measure and one finds: 


o is a gain equilibrium of U <=> o is a cost equilibrium of C 


‘cf. Chapter 8. 
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From an abstract point of view, the theory of gain equilibria is 
equivalent to the theory of cost equilibria. 


Many real-world systems appear to evolve in dynamic processes 
that eventually settle in an equilibrium state (or at least approximate 
an equilibrium) according to some utility measure. This phenomenon 
is strikingly observed in physics. But also economic theory has 
long suspected that economic systems may tend towards equilibrium 
states.” 


2.1. Existence of equilibria 


In practice, the determination of an equilibrium is typically a very 
difficult computational task. Moreover, many utilities do not even 
admit equilibria. It is generally not easy just to find out whether an 
equilibrium for a given utility exists at all. Therefore, one is interested 
in manageable conditions that allow one to conclude that at least one 
equilibrium exists. 


Utilities from potentials. Consider a utility potential u: G > R 
with the marginal utility measure 


Ou(o,T) = u(r) — u(o). 


Here, the following sufficient conditions offer themselves immediately: 


OS Beek Won) max u(r), then o is a gain equilibrium. 


(2) alin min u(r), then o is a cost equilibrium. 
ie 


Since every function on a finite set attains a maximum and a 
minimum, we find 


PROPOSITION 5.2. If G is finite, then every utility potential 


yields a utility measure with at least one gain and one cost 
equilibrium. 


2A.A. COURNOT (1838-1877) [9]. 
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Similarly, we can derive the existence of equilibria in systems that 


are represented in a coordinate space. 


PROPOSITION 5.3. If G can be represented as a compact set 
S CR” such that u: S > R is a continuous potential, then u 


yields a utility measure Ou with at least one gain and one cost 
equilibrium. 


Indeed, it is well-known that a continuous function on a compact 
set attains a maximum and a minimum. 


REMARK 5.2. Notice that the conditions given in this section are 
sufficient to guarantee the existence of equilibria — no matter what 
neighborhood structure on G is assumed. 


Convex and concave utilities. If the utility measure U under 
consideration is not implied by a potential function, not even the 
finiteness of G may be a guarantee for the existence of an equilibrium 
(see Ex. 5.2). 


Ex. 5.2. Give the example of a utility measure U relative to a finite 
state set G with no gain and no cost equilibrium. 


We now derive sufficient conditions for utilities U on a system 
whose set G of states is represented by a nonempty convex set 
S CR”. We say: 


e U is convex if every local function u* : S > R is convex. 
e U is concave if every local function u* : S > R is concave. 


THEOREM 5.1. Let U be a utility measure with continuous 
local utility functions u® : S + R on the nonempty compact 
set S CR™. Then 


(1) If U is convex, a cost equilibrium exists. 
(2) If U is concave, a gain equilibrium exists. 
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Proof. Define the function G:S x S — R with values 
G(s,t)=u*(t) for all s,t eS. 


Then the hypothesis of the Theorem says that G satisfies the 
conditions of Corollary A.1 of the Appendix. Therefore, an element 
8s € S exists such that 


u*(t) = G(s,t) < G(s,s) =u*(s) holds for all s € S. 


Consequently, s* is a gain equilibrium of U. (The convex case is 


proved in the same way.) 
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Chapter 6 


n-Person Games 


m-person games generalize 2-person games. Yet, it turns out that the 
special techniques for the analysis of 2-person games apply in this 


seemingly wider context as well. Traffic systems, for example, fall into 
this category naturally. 


The model of an n-person game IT assumes the presence of a finite 
set N with n = |N| elements together with a family 


X¥ ={X,;|ieN} 


of n further nonempty sets X;. The elements i € N are thought of as 
players (or agents, etc.). A member X; € ¥ represents the collection 
of resources (or actions, strategies, decisions, etc. ) that are available 
to agent 7 € N. 

A state of T is a particular selection x = (a; | i € N) of individual 
resources x; € X; by the n agents 7. So the collection of all states x 
of I is represented by the direct product 


x= || x 
ieN 
It is further assumed that each player i € N has an individual utility 
function 


Uuz:xX7R 
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by which its individual utility of any x € X is assessed. The whole 
context 


T=T(u;|ie N) 
now describes the n-person game under consideration. 


Ex. 6.1. The matrix game [I with a row player R and a column 
player C' and the payoff matrix 


_ (pi, M11) eee fe hie (—1,+1) 
(p21,921) (p22, q22) Lely), Ala) 


is a 2-person game with the player set N = {R,C} and the strategy 
sets Xp = {1,2} and Xc = {1,2}. Accordingly, the set of states is 


P= pe Xe = fA Mya 25 2}. 
The individual utility functions ur, uc : X > R take the values 
ur(s,t)=pse and uc(s,t)=aqse for all (s,t) € X. 


REMARK 6.1. It is often convenient to label the elements of N 
by natural numbers and assume N = {1,2,...,n} for simplicity of 
notation. In this case, a state x of I can be denoted in the form 


MS (B15 Woy wey) SAX Ao ee K A 


Cooperation. The basic game model with a set N of players is 
readily generalized to a model where groups of players (and not just 
individuals) derive a utility value from a certain state x € X. To this 
end, we call a subset S C N of players a coalition and assume an 
individual utility function ug : X > R to exist for each coalition S. 

From an abstract mathematical point of view, however, this 
generalized model can be treated like a standard |N/|-person game, 
having the set 


N ={SCN} 


of coalitions as its set of “superplayers”. In fact, we may allow 
each coalition S to be endowed with its own set Xg of resources. 
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In this chapter, we therefore retain the basic model with respect to 
an underlying set N of players. 

Further aspects come to the fore, however, when one asks what 
the strategic decisions at coalition level mean for the individual 
players. For example: 


e How should one assess the power of an individual player? 
e How do coalitions come about? 


A special class of potential n-person games with cooperation, so- 
called TU-games, will be studied in their own right in more detail in 
Chapter 8. 


Probabilistic models. There are many probabilistic aspects of 
n-person games. One consists in having a probabilistic model for 
the choice of actions to start with (see Ex. 6.2). 


Ex. 6.2 (Fuzzy games). Assume a game [ where any player i € N 
has to decide between two alternatives, say “0” and “1”, and chooses 
“1” with probability z;. Then T is an |N|-person game in which each 
player 7 has the unit interval 


X,=(0,1J={eeR|0<2<1} 
as its set of resources. A joint strategic choice 
X= (By. +2405, s.450n) € (0, 1)* 
can be interpreted as a “fuzzy” decision to form a coalition X C N: 
e Player 7 will be a member of X with probability 2;. 


x is thus the description of a fuzzy coalition. T is a fuzzy cooperative 
game in the sense of AUBIN [1]. 


A further model arises from the randomization of an n-person 
game (see Section 3 below). Other probabilistic aspects of n-person 
games are studied in Chapter 8 and in Chapter 9. 
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1. Dynamics of n-person games 


If the game T = (u; | i € N) is played, a game instance yields a 
sequence of state transitions. The transitions are thought to result 
from changes in the strategy choices of the players. 

Suppose 7 € WN replaces its current strategy x; by the strategy 
y € X; while all other players 7 4 7 retain their choices 7; € Xj. 
Then a state transition x + y = x_;(y) results, where the new state 
has the components 


Two neighboring states x and y differ in at most one component. In 
particular, x_;(x;) = x holds under this definition and exhibits x as 
a neighbor of itself. Let us take the set 


Fi(x) = {x-ily) | y © Xi} 


as the neighborhood of the state x € X for the player i € N. So 
the neighbors of x from 7’s perspective are those states that could be 
achieved by 7 with a change of its current strategy x;, provided all 
other players 7 #7 retain their current strategies x;. 

The utility functions u; thus provide the natural utility measure 
U for T with the values 


U(x,y) = ui(y) — u(x) for allie N and x,y € F;(x). (41) 


Potential games. The n-person game [ = (u; | 7 € N) is called a 
potential game if there is a potential v : X > R such that, for all 
i € N andx,y ©€ F;(x) the marginal utility change equals the change 
in the potential: 


uily) — ui(x) = v(x, y) = o(y) — v(x). (42) 


2. Equilibria 


An equilibrium of T = T(u; | 7 € N) is an equilibrium of the utility 
measure U as in (41). The joint strategic choice x € & is thus a gain 
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equilibrium if no player has an utility incentive to switch to another 
strategy, 7.e., 


uj(x) > u(y) holds for all i € N and y € F;(x). 


Completely analogously, a cost equilibrium is defined via the reverse 
condition: 


uj(x) < u(y) holds for all i€ N and y € F;(x). 


Aggregated utilities. There is another important view on equilib- 
ria. Given the state x € X, imagine that each player i € N considers 
an alternative y; to its current strategy x;. The aggregated sum of 
the resulting utility values is 


G(x,y) = do ualy)) (= (vi ly € Xi). 
tEN 


LEMMA 6.1. x € & is a gain equilibrium of T(u; | i € N) if 
and only if 


G(x, y) < G(x,x) holds for ally € &. 


Proof. If x is a gain equilibrium and y = (y; | i € N) € X, we have 
uj(x) 2 u(x_i(y)) for all yy € Xi, 


which implies G(x, x) > G(x,y). Conversely, if x is not a gain equi- 
librium, there is anz € N and ay € X; such that 


0 < ui(xi(y)) — wilx) = G(x, xi(y)) — G(x, x). 


which means that y = x_;(y) € ¥ violates the inequality. 


Lemma 6.1 reduces the quest for an equilibrium to the quest for a 
x € X that maximizes the associated component function g* :X > R 
of the aggregated utility measure G with the values 


go (y) = G(x, y). 


It follows immediately that we can carry over the general 
sufficient conditions in Chapter 5 for the existence of equilibria to the 
n-person game [ = (u; | i € N) with utility aggregation function G: 
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(1) If [ is a potential game with a finite set X of states, 
then the existence of a gain and of a cost equilibrium is 
guaranteed. 


If X is represented as a nonempty compact and convex set 


in a finite-dimensional real parameter space, and all the 
maps y +> G(x,y) are continuous and concave, then [ 
admits a gain equilibrium. 

If X is represented as a nonempty compact and convex 
set in a finite-dimensional real parameter space, and all 
the maps y +> G(x, y) are continuous and convex, then [ 
admits a cost equilibrium. 


Ex. 6.3. Show that the matrix game in Ex. 6.1 is not a potential 
game. (Hint: The set of states is finite.) 


3. Randomization of matrix games 


An n-person game [ = (u; | i € N) is said to be a (generalized) 
matriz game if all individual strategy sets X; are finite. 

For a motivation of the terminology, assume N = {1,...,n} 
and think of the sets X; as index sets for the coordinates of a 
multidimensional matrix U. A particular index vector 


X= (ies) EAL Ker KAG Ke Ry es) 
thus specifies a position in U with the n-dimensional coordinate entry 
Ue =i GOs. wa a(S) ER”, 
Let us now change the rules of the matrix game [ in the following 


way: 


(R) Each player i € N chooses a probability distribution p 
(i) 


on X; and selects the element x € X; with probability pz’. 
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Under rule (R), the players are really playing the related n-person 
game I’ = (u; |i € N) with resource sets P; and utility functions Uj, 
where 


(1) P; is the set of all probability distributions on X;. 


(2) %;(p) is the expected value of u; relative to the joint pro- 
bability distribution p = (p | i € N) of the players. 


The n-person game I is the randomization of the matrix game 
T. The expected total utility value is 


r1EX4 tInEXn 


As Ex. 6.1 shows, a (non-randomized) matrix game [ does not 
necessarily have equilibria. Notice, on the other hand, that the 
coordinate product function 


(t1,...,tn) ER" 4 ty---t, ER 


is continuous and linear in each variable. Each utility function @; of 
the randomized game T is a linear combination of such functions and, 
therefore, also continuous and linear in each variable. 

Since linear functions are both concave and convex and the state 
set 


B= Px---x Pr 


of T is convex and compact, we conclude: 


THEOREM 6.1 (NASH [30]). The randomization T of an 


n-person matrix game IT admits both a gain and a cost 
equilibrium. 
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REMARK 6.2. An equilibrium of a randomized matrix game is also 
known as a Nasu! equilibrium. 


4. Traffic flows 


A fundamental model for the analysis of flows in traffic networks goes 
back to WARpDROP [45]. Let us look at a discrete version? of it. It is 
based on a graph G = (V,£) with a (finite) set V of nodes and set 
E of (directed) edges e between nodes, 


O>®) 
representing directed connections from nodes to other nodes. The 
model assumes: 


(W) There is a set N of players. A player 7 € N wants to travel 


along a path in G from a starting point s; to a destination 
t; and has a set P; of paths to choose from. 


Game-theoretically speaking, a strategic action s of player 1 € N 
means a particular choice of a path P € P;. Let us identify a path 
P € P; with its incidence vector in R® with the components 


1 if P passes through e 
©“) 0. otherwise. 


The joint travel path choice s of the players generates the traffic flow 


aye ARP of size Ix®| = SOAR <n, 
P 


tEN PEP; 


where A} is the number of players that choose path P in s. The 
component 2% of x® is the amount of traffic on edge e caused by the 
choice s. 


'J.F. NASH (1928-2015). 
2Due to ROSENTHAL [36]. 
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We assume that a traffic flow x produces congestion costs c(2-) 
along the edges e and hence results in the total congestion cost 


Cle): = S> Gal He lie 


e€cE 


across all edges. An individual player i has the congestion cost just 
along its chosen path P: 


CRs) = Ss" Gel te): 


e€P 


If we associate with the flow x the potential of aggregated costs 


B(x) = Deel), 


e€E t=1 


we find that player 2’s congestion cost along path P in x equals the 
marginal potential: 


C(P,x) = ¥ > ce(te) = B(x) — O(x — P) = Op ®(x — P). 
e€P 


It follows that the players in the WARDROP traffic model play an 
n-person potential game on the finite set X of possible traffic flows. 

x € X is said to be a NASH flow if no player 7 can improve its 
congestion cost by switching from the current path P € P; to the 
use of another path Q € P;. In other words, the NASH flows are the 
cost equilibrium flows. Since the potential function ® is defined on a 
finite set, we conclude 


e The WARDROP traffic flow model admits a NASH flow. 


BRAESS’ paradox. If one assumes that traffic in the WARDROP 
model eventually settles in a NASH flow, i#.e., that the traffic 
flow evolves toward a congestion cost equilibrium, the well-known 
observation of BRAESS [6] is counter-intuitive: 
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(B) It can happen that a reduction of the congestion along 


a particular connection increases(!) the total congestion 


cost. 


As an example of BRAESS’ paradox, consider the network G = 
(V, E) with 


V= {8, rT, q; ey and E= {(s, r), (s, q); (r, t); (q, i); (r, q)}- 
Assume that the cost functions on the edges are 
Guo) =a egtt) =A.) SA ee) =a. =H 10 


and that there are four network users, which choose individual paths 
from the starting point s to the destination t and want to minimize 
their individual travel times. 

Because of the high congestion cost, no user will travel along 
r,q). As a consequence, a NASH flow will have two users of path 
P = (s > r > t) while the other two users would travel along 
P=(s>q-t). 

The overall cost is: 


COP 19P)HO09 4-0 4594979 94, 


If road improvement measures are taken to reduce the congestion on 
(r,q) to Gri = 0, a user of path P can lower its current cost C(P) = 6 
to C(Q) = 5 by switching to the path 


Q=(s7~r>q-ty). 
The resulting traffic flow, however, causes a higher overall cost: 


CO(P4Q 49P) j= 224 45143-14452 3.9 = 95) 


Chapter 7 


Potentials and Temperature 


The temperature of a system depends on the measuring device in use, 
which is mathematically represented as a potential function. BOLTZ- 
MANN’s approach to the notion of temperature in statistical thermody- 
namics extends to general systems. Of particular interest are n-person 
matrix games where the temperature reflects the activity of the player 


set as a whole with respect to the total utility. The interpretation 
of the activity as a METROPOLIS process moreover indicates how the 
strategic decisions of individual players influence the expected value of 
the measuring device. 


Consider a finite system © that is in a state o with probability z,. 
Then G has the entropy 


Ha) = S > to In(1/7.) =— So te Int. 


o€y coed 


The expected value of a potential v € R© will be 


Ewe ary = Ss" UisTts 
oE&G 

Let us think of v as a numerical measuring device for a certain 
characteristic feature of G. In a physical model, the number vu, 
could describe the level of inherent “energy” of G in the state o, 
for example. In economics, the function v : G — R could be a 
representative statistic for the general state of the economy. In the 
context of an n-person game, v, could possibly measure a degree of 
“activity” of the set N of players in the state o, etc. 
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Of course, the activity level v, depends on the particular 
characteristic feature that is investigated under v. Different features 
of G may display different activity levels in the same state o. 


1. Temperature 


If the expected value » = E(v,7) is a measure for the degree 
of activity of G relative to the probability distribution 7 and the 
potential v, there may nevertheless be other probability distributions 
m’ with the same expected value 


E(v,n) = p= E(v,7’). 


The idea is now to derive a canonical probability distribution @ with 
a given expected value yw. To this end, we select 6 as the distribution 
with the largest entropy: 


A(8) = max{H(r) | E(v,7) = py}. 


As it turns out (Lemma 7.1 below), every other probability distribu- 
tion a yielding the same expected value yz of v will have a strictly 
smaller entropy H(z) < H(G), which means that its specification 
would require more information. In this sense, ( is the unique “freest” 
(i.e., least biased) distribution with expectation p. 


1.1. BOLTZMANN distributions 


Given the potential v : G — R with values v, = v(o), any t EC R 
defines a related BOLTZMANN! (probability) distribution B(t) on G 
with the components 


Ba(t) — Z where Z(t) = ae > 0. 


REMARK 7.1. The distributions §(t) are also known as 
BOLTZMANN-GIBBS? distributions. The function Z(t) is the asso- 
ciated so-called partition function. 


'L. BOLTZMANN (1844-1906). 
2 J.W. Gibbs (1839-1903). 
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Computing derivatives, one finds® that the expected value of v 
relative to b(t) can be expressed as the logarithmic derivative of the 
partition function: 


2 Yrs 7 a = = (43) 


Nota BENE. Under a BOLTZMANN distribution B(t) with t € R, 
no state of G will be impossible (i.e., occur with probability 0) and no 
state will occur with certainty (i.e., with probability 1) if G has more 
than one state. 

In the special case t = 0, one has Z(0) = |G|. So 6(0) is the 
uniform distribution on G with the average potential value as its 
expectation: 


= TBI a 


cEG 


Moreover, one observes the limiting behavior 


In fact, one finds that any value yw between the minimum and 
the maximum of v can be achieved as the expected value relative 
to a unique BOLTZMANN distribution. Moreover, the BOLTZMANN 
distribution is the one with the largest entropy relative to a 
prescribed expected value: 


LEMMA 7.1. Let vy = minges and v* = maxgce Vo. Then: 


(Ong 7 a i alas ror ant IR 
(1) For every vx < pp < v*, there exists a unique t € R such 


that = p(t). 
(2) H(m) < H(B(t)) holds for every probability distribution 
nt # B(t) on G mit expectation E(v, 7) = p(t). 


3See also Section 6 of the Appendix. 
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1.2. BOLTZMANN temperature 


From Lemma 7.1, it is clear that one could characterize the expected 
value ys of a non-constant potential v equally well by specifying the 
parameter t € RU {—00, +00} of the BOLTZMANN distribution [(t) 
with expectation 


In analogy with the BOLTZMANN model in statistical thermodynam- 
ics for the temperature, we call the related parameter 


Pit 


the temperature of the system © relative to a potential with the 
expected value y(1/T). Adjusting the notation accordingly to 


BY = p/T) and pl = p(1/T). 
the BOLTZMANN distribution 8) has the coefficients 


gl?) = ae 
res eel 


As the system “freezes” to the temperature JT’ = 0, one obtains 
the extreme values of the potential v as the expectations in the limit, 


(c € G6). 


depending on whether the limit 0 is approached from the positive or 
the negative side: 


lim pi) = MAX Yo 
o€ 


T0+ 
lim py?) =mintg. 
T-0- acG 


In contrast, all states of G are equally likely at when the 
temperature J’ is infinite. 
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2. The METROPOLIS process 


METROPOLIS et al. [29] have pointed out that BOLTZMANN distribu- 
tions may result dynamically as limiting distributions of quite general 
stochastic processes. 

To formulate the process, we associate with each o0 € Ga 
neighborhood F(c) C G that renders G connected in the following 
sense: 


e For any o,7 € ©, there are o1,...,0% € © such that 
oe41 € F(ove) holds for all 2=0,...k, 


where o9 = o and og41 =T. 


The METROPOLIS process specifies a parameter t € R and 
proceeds from an initial state o9 in a (possibly infinite) 
sequence of state transitions relative to the potential v in the 
following way. 


When the process has reached the state a, a neighbor rT € F(c) 
is selected at random as a candidate for a transition 0 > T 
and then proceeds as follows: 


(1) If v, > vo, then the transition 0 — 7 is made with 
certainty, 7.e., with probability 


dor = IL. 


(2) If vu; < vo, then the transition o — 7 is made with 
probability 


dor = Aa eelll 


and repeated otherwise. Hence the transition selection 
procedure is repeated with probability 1 — qg-. 
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It follows that the METROPOLIS state transitions define a 
MarRKOV chain‘ on © with transition probabilities 


per = 82 
IF()| 


The METROPOLIS process converges as a MARKOV chain to a 
limiting distribution on G under quite general conditions. A sufficient 
condition is, for example: 


PROPOSITION 7.1. Assume that all neighborhoods F(a) have 
the same size f = |F(c)|. Then the METROPOLIS process 
converges to the BOLTZMANN probability distribution B(t) in 
the limit. 


In other words: The process will arrive at the state o € © after 
n iterations with probability 


pr) > Bo(t) (asn—- oo). 


Proof. By Ex. A.12 of Section 7 of the Appendix, it suffices to check 
that the condition 


Bo (t) Dor = B, (t)Pro : 


is satisfied by any o,T € G. So assume v, < vg, for example. Then 


evote(vr—Uc )t curt 


Bo(t)Por = — Zor = Z(Hf 


= B, (t) Dro . 


Simulated annealing. The METROPOLIS process suggests a simple 
intuitive method for maximizing a function v: X — R over a finite 
set X: 


max Vy. 
rEx 


“See Section 7 in the Appendix. 
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(So) Associate with each x € X a neighborhood set F(x) and 
designate an initial element xp € X. 

(S;) Choose a sequence to, t1,...,tn,... of natural numbers t,, 
with t, + oo. 

(S,,) Iterate the following procedure for n = 0,1,2,...: 


(1) If x, € X has been constructed, choose a neighbor 


y € F(a) randomly as a candidate for the next 


element 241. 
Ue cet ele 
(3) If vy < vz, set 
_ { y with probability e(@y—ven)t 
oe a with probability 1 — e(@y—Yen)tn, 


There is no guarantee that the simulating annealing heuristic will 
find a maximizer of v. However, the analogy with the METROPOLIS 
procedure makes it intuitively plausible that the method might 
generate elements x, with the property 


Uz, © U(tn) > mMaxvp, (n— ov). 
rex 
The schematic description of the simulated annealing heuristic 
leaves many implementation details vague. For example: 


e What is the best neighborhood structure in (So)? 
e How should the numbers to, t,,... in (S;) be chosen best? 


Nevertheless, the method often produces quite good results in 
practice? and has become a standard in the tool box for discrete 
optimization problems. 


REMARK 7.2. The terminology “annealing” (= cooling) derives 
from the interpretation of the related parameters T, = 1/tn as 
temperatures which simulate a cooling schedule T;, > 0. 


See, e.g., S. KIRKPATRICK et al. [26]. 
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REMARK 7.3. The import of the simulated annealing method is 
not so much the generation of BOLTZMANN distributions at fixed 
temperature levels but the convergence of the expected potential 
values y(t) to the extremum: 


t) -— io t— ; 
u(t) + maxvs (as t + 00) 


The latter property can be guaranteed under much more general 
conditions than are formulated in Proposition 7.1.° 


3. Temperature of matrix games 


Let T = ['(u; | i € N) be an n-person game with player set N = 
{1,...,} where each player i € N has a finite set X; of strategic 
resources and a utility function 


u:X—3>R (with X= X) x X.x...x Xn). 


In the model of randomized matrix games, it is assumed that the 
players i choose probability distributions 7 on their strategy sets 
X;, independently from each other and then select elements x; € X; 
according to those distributions. 

Let us drop the stochastic independence assumption and consider 
the more general model where the joint strategy 


= (Dita sea 


would be chosen by the player set N with a certain probability 7x. 
The aggregated total utility value is then expected to be 


b= SS De Uj(X) Tx. 


xEXiEeN 


The players’ total utility 


ax) = Ss" uj (x) 


iEN 


®See, e.g., U. FAIGLE and W. Kern [14]. 
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is a potential on X. So one may consider the (BOLTZMANN) temper- 
ature relative to u. In the case 


1 
p= F- So eMV/T (with Zp = Z(1/T)) 
xEX 


we say that [ is is played at temperature T. If |T'| = 00 (i.e., |T| is 
very large), we expect about the average value of the total utility: 


1 
pb & ee UO: 


xEX 


If T > 0 is very small (i.e., T + 0), then we may expect about 
the maximal total utility: 


ae 


Similarly, if T + 0 and T < 0 holds, about the minimal total 
utility value is to be expected: 


w= min u(x). 


A METROPOLIS model. In order to model the dynamic behavior 
of the player set N as a METROPOLIS process, define the set 


v= {(i,y) |b N,y € XG}. 


A selection (i,y) € *, when the game is in the state x € &, 
corresponds to a possible transition 


x + x_i(y) 


with the interpretation that player 7 considers the alternative 
strategy y € X; as a candidate for an exchange against the current 
x; € X;. Hence, if (7, y) is chosen randomly, the probability for the 
corresponding transition is 


Prfoe_a(w))} = Faz min{ 1, eK /T, 
So each x € X has the same number f = || of neighbors. Clearly, 
this neighborhood structure is connected. Hence Proposition 7.1 
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shows that the process converges to a BOLTZMANN probability 
distribution: 


(n) eux) /T 
Be yer 
yex 


Recall that every state x occurs with strictly positive probability in 
a BOLTZMANN distribution at temperature T 4 0. So we should not 
expect convergence towards an equilibrium with respect to u. 

This observation does not contradict the guaranteed existence 
of an equilibrium in the randomized version of a matrix game 
(Theorem 6.1). The reason can be found in the different roles of 
the players in the models: 


(1) In a randomized matrix game, each player i € N chooses, 
individually, a probability distribution 7 for the selection a 
a; € X; with the goal of optimizing the individual utility u;. 

(2) In the METROPOLIS process, the total utility u of the group N 
of players is the key objective. 


REMARK 7.4 (Social justice). It appears to be in the joint 
interest of the community N of players to play I’ at a tempera- 
ture T that is close to 0 but positive if a large total utility value 
is desired and negative if a minimal value is sought. 


The potential function u is equivalent (up to the scaling factor 
n) to the average utility function @ of the members of N: 


(Se) S/ ui(x) <> Ux)= = So u(x). 
j=l i=1 


A high group average does not necessarily imply a guaranteed 
high utility value for each individual member in N, however. 


To formulate it bluntly: 


e Even when a high average utility value is used as a 
criterion for “social justice” in N, there may still be 
members of N that are not treated “fairly”. 
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The interplay of different interests (individual utility of the 
players vs. combined utility of the set of all players) is studied 
in more details within the framework of cooperative games in 
Chapter 8. 


This page intentionally left blank 


Chapter 8 


Cooperative Games 


Players in a cooperative game strive for a common goal, from which 
they possibly profit. Of special interest is the class of TU-games with 
a transferable utility potential, which is best studied within the context 
of linear algebra. Central is the question how to distribute the achieved 
goal’s profit appropriately. The core of a cooperative game is an important 
analytical notion. It strengthens the VON NEUMANN-MORGENSTERN 


solution concept of stable sets and provides a link to the theory of discrete 
optimization and greedy algorithms. It turns out that the core is the only 
stable set in so-called supermodular games. Values of cooperative games 
are more general solution concepts and can be motivated by stochastic 
models for the formation of coalitions. Natural models for the dynamics 
of coalition formation are closely related to thermodynamical models in 
statistical physics and offer an alternative view on the role of equilibria. 


While the agents in the n-person games of the previous chapters 
typically have individual utility objectives and thus possibly opposing 
strategic goals, the model of a cooperative game refers to a finite set N 
of n = |N| players that may or may not be active towards a common 
goal. A subset S C N of potentially active players is traditionally 
called a coalition. Mathematically, there are several ways of looking 
at the system of coalitions: 


From a set-theoretic point of view, one has the system of the 2” 
coalitions 


N ={S|SCN}. 
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On the other hand, one may represent a subset S € N by its incidence 
vector x5) € RN with the coordinates 


S.A 
“= )0 ift€S. 


S) 


The incidence vector x°) suggests the interpretation of an “activity 


vector”: 


(S) 4, 


1€ N is active if x; 


The coalition S' would thus be the collection of active players. 

A further interpretation imagines every player 7 € N to have a 
binary strategy set X; = {0,1} from which to choose one element. 
An incidence vector 


w= (£1,...,2n) € X1x---x Xp, = {0,1}% CR 


represents the joint strategy decision of the n players and we have 
the correspondence 


N = > 40,1)" =2" 


By a cooperative game we will just understand a n-person game I" 
with player set N and state set 


X=N or €=2N, 


depending on a set-theoretic or on a vector space point of view. 
A general cooperative game T= (u; | i € N) with individual utility 
functions u; : NV — R is therefore a matrix game where each player 
has the choice between two alternative actions. 

In the following, we will concentrate on cooperative games whose 
individual utilities are implied by a general potential on NV. 


1. Cooperative TU-games 


A transferable utility relative to a set N of players is a quantity v 
whose value v(S) depends on the coalition S of active players and 
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hence is a potential 
vu:N OR. 


The potential v implies the utility measure Ov : N x N > R with 
the values 


Ov(S,T) = v(T) — v(S). 


We denote resulting potential game by [ = (N,v) and refer to it as 
a cooperative TU-game with characteristic function v. 


Ex. 8.1. Assume that the players i € WN evaluate their utility 
relative to a coalition S C N by a real parameters u;(S), which 
means that each player 7 € N has an individual utility function 


ujiNoOR. 


The aggregated utility u = }?,-, ui is then a transferable utility and 
defines the TU-game [ = (N,u). For each S C N and player i € N, 
one finds: 
Ou;(S,S U {t}) = uj(S U {2}) — u;(S) 
= u(S'U {i}) — u(S) = du(S, SU {i}). 
REMARK 8.1 (Terminology). Often the characteristic function v 


of a cooperative TU-game (JN, v) is already called a cooperative game. 
In discrete mathematics and computer science a function 


v:{0,1}" —>R 


is also known as a pseudo-boolean function. Decision theory refers to 
pseudo-boolean functions as set functions.! 


The characteristic function v can represent a cost utility or 
a profit utility. The real-world interpretation of the mathematical 
analysis, of course, depends on whether a cost or a gain model is 
assumed. Usually, the modeling context makes it clear, however. 


‘See, e.g., GRABISCH [21]. 
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Equilibria. Since WV is a finite set, a TU-game (N,v) has equilibria. 
Indeed, any coalition S with maximal value 


v(S) = max o(T) 


is a gain equilibrium, while a minimizer of v is a cost equilibrium. 


Normalizations. We will always assume that the characteristic 
function v of a TU-game is zero-normalized in the sense 


v(O) = 0. (45) 


The assumption (45) is made for mathematical convenience. In the 
case u() ~ 0, one may study the related zero-normalized game 
(N,v’) with 


v'(S) = v(S) — v(@). 


Essential features of (N,v) are shared by (N,v’). v and v’ imply the 
same utility measure, for example. In this sense, zero-normalization 
does not result in a loss of generality in the mathematical analysis. 

Other normalizations are sometimes useful. Setting, for example, 


v"(S) = v(S) — So o({i} for all SEN, 
iEeS 
one obtains a related game (N,v”) where the individual players i € N 
have the potential value v’({i}) = 0. 


In the sequel, we will concentrate on (zero-normalized) TU-games 
and therefore just talk about a cooperative game (N,v). 


2. Vector spaces of TU-games 


Identifying a TU-game (N,v) with its characteristic function v, we 
think of the function space 


RY ={v:N—>R} with N={SCN} 


as the vector space of all (not necessarily zero-normalized) TU-games 
on the set N. RY is isomorphic with coordinate space R2” and has 
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dimension 
dim RY = |N| = 2” = dimR?”. 


The 2” unit vectors of RY correspond to the so-called DIRAC 
functions 65 € RY with the values 


1 fT=S 
os) = ‘i ifT 4S. 


The set {6g | S € N} is a basis of RV. Any v € R™ has the 
representation 


v= S° v(S)és. (46) 


SEN 


2.1. Duality 


It is often advantageous to retain NV as the index set explicitly in the 
game-theoretic analysis and make use of the set-theoretic structure of 
the coalition ensemble VV. One such example is the duality operator 
v + v* on RY, where 


v*(S) =v(N)—v(N\ 8) for all SCN. (47) 


We say that the game (N,v*) is the dual of (N,v). For any possible 
coalition S € NV’, the numerical value 


u"(N \ S) = v(N) — v(S) = Ov(S, N) 


is the “surplus” of the so-called grand coalition N vs. S in the game 
(N,v). So duality expresses a balance 


v(S) + u*(N \ S) =v(N) for all coalitions S. 
Ex. 8.2. Show: 


(1) v + v* is a linear operator on RY. 
(2) The dual v** = (v*)* of the dual v* of uv yields exactly the zero- 
normalization of v. 


138 &. Cooperative Games 


2.2. MOBIUS transform 


For any v € RW, let us define its MOBIus? transform as the function 
6 € RY with values 


a(S) = Sof) (SEN). (48) 


TCS 


6(S) aggregates the v-values of all subcoalitions T C S. In this sense, 
the MOBIUS transformation v +> 0 is a kind of “discrete integral” on 
the function space RY. 


Ex. 8.3 (Unanimity games). The MOBIUS transform dg of 
the DiRAC function dg is known as a unanimity game and has 
the values 


6s(T) = 


Be eee es 


0 SCT. 


A coalition T has a non-zero value 63(T) = 1 exactly when 
the coalition T includes all members of S. Unanimity games 
appear to be quite simple and yet are basic (Corollary 8.1 
below). Many concepts in cooperative game theory are tested 
against their performance on unanimity games. 


Clearly, the MOsIus transformation is a linear operator on 
RY. The important observation concerns an inverse property: every 
characteristic function uv arises as the transform of a uniquely 
determined other characteristic function w. 


THEOREM 8.1 (MOBIUS INVERSION). For each v € RY, 


there is a unique w € R™ such that v = w. 


2A.F. MOstus (1790-1868). 
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Proof. Recall from linear algebra that, in view of the linearity of 
the MOBIUS transformation, it suffices to show: 


zZ=O z=O, 


i.e., the kernel of the map v +> @ contains just the zero vector 
OER. 

So assume that z € RY transforms to # = O. Let S € N bea 
coalition and observe in the case S = 0): 


2(0) = 2(0) =0. 


Assume now, by induction, that z(7’) = 0 holds for all T € N of size 
|T| < |S|. Then the conclusion 


2(S) = 4(S)- S° 2(f) =0-0=0 
TCS 


follows and completes the inductive step of the proof. So z(S) = 0 
must be true for all coalitions S. 


Since the MOBIUS operator is linear, Theorem 8.1 implies that it is, 
in fact, an automorphism of RY , which maps bases onto bases. In 
particular, we find: 


COROLLARY 8.1 (Unanimity Basis). The unanimity 
games Os form a basis of RN, ie., each v € RY admits a 
unique representation of the form 


ve SE Neve with coefficients Ag ER. 
SEN 


Ex. 8.4 (HARSANYI dividends). Where v = wW, the values w(S) 
are known as the HARSANYI dividends of the coalitions S in the game 
(N,v). It follows that the value v(S') of any coalition S is the sum of 
the HARSANYI dividends of its subcoalitions 7’: 
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REMARK 8.2. The literature is not quite clear on the terminology 
and often refers to the inverse transformation ¢ +» v as the 
MOBIUS transformation. Either way, the MOBIuSs transformation 
is a classical and important tool also in in number theory and in 
combinatorics.° 


2.3. Potentials and linear functionals 


A potential f : NV — R, interpreted as a vector f € R% defines a 
linear functional f : RY — R with the values 


F(9) = (fla) = 5 fsgs for all gE RN. 
SEN 


If g°) is the (0, 1)-incidence vector of a particular coalition S € N, 
we have 


f(g) = (flg™) = fs 1= fs, 


which means that f extends the potential f on 2% (= N) to all 
of RN. 

Conversely, every linear functional g ++ (fg) on R defines a 
unique potential f on N via 


f(S) = (flg) for all SEN. 
These considerations reveal characteristic functions on NV and linear 


functionals on R to be two sides of the same coin. From the point 
of view of linear algebra, one can therefore equivalently define: 


e A cooperative TU-game is a pair = (N,v), where N is a 


set of players and v ++ (v|g) is a linear functional on the 
vector space RN. 


3See, e.g., ROTA [37]. 
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2.4. Marginal values 


The characteristic function v of the cooperative game (N,v) implies 


the utility measure* 


Ov(S,T) = v(T) — v(S). 


Individual players 7 € N will assess their value with respect to v by 
evaluating the change in v that they can effect by being active or 
inactive. 

For a player 7 € N, we thus obtain its marginal value with respect 


to the coalition S as 


Ex. 8.5 (Symmetric difference). The symmetric difference of 
two sets S,T is the set 


SAT =(S\T)U(T\ 8) =(SUT)\ (SOT). 


With this notion, the marginal value of player i € N in (N,v) 
becomes 


8;(S) = v(SAi) — o(S) = av(S, SAd). 


3. Examples of TU-games 
3.1. Additive games 


The marginal value 0;v(S') of a player i depends on the coalition S$ 
it refers to. Different coalitions may yield different marginal values 
for the player 7. 

The TU-game (N,v) is said to be additive if a player would add 
the same marginal value to each coalition it joins. This means: There 


4See also Ex. 8.1. 
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is a number v; for each 7 € N such that for all coalitions S C N with 


1S, 


Hence, if v is zero-normalized and additive, one has 


uS)'= yo: 


ieS 


Conversely, every vector a € R% defines a unique zero-normalized 
additive game (N,a) with the understanding 


a(@) =0 and a(S) = Sa for all SA 0. (49) 


ses 


Ex. 8.6. Which unanimity games (see Ex. 8.3) are additive? Show 
that the vector space of all additive games on N has dimension 
|N| + 1. The subspace of all zero-normalized additive games on N 
has dimension ||. 


3.2. Production games 


Similar to the situation in Section 4.3, consider a set N of players 
in an economic production environment where there are m raw 
materials, M,,...,M,, from which goods of k different types may 
be manufactured. 

Let x = (a1,...,2,%) be a plan that proposes the production of 
x; > 0 units of the jth good and assume: 


(1) x would need a;(x) units of material M; for all i = 1,...,m; 

(2) each supplier s € N has bj, > 0 units of material M; at its 
disposal; 

(3) the production x could be sold for the price of f(x). 


So the coalition S C N could guarantee a production of market value 


vS) = max flr) sts ale)s S bis (ede ym): (50) 


k 
eoRy ses 


The corresponding cooperative TU-game (N,v) is a production game. 
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What is the worth of a player? This is one of the central ques- 
tions in cooperative game theory. In the context of the production 
game (N,v), one natural approach to resolve this question is the 
market price principle: 


(MP) Assuming that each material M; has a market price of y; 


per unit, assign to each supplier s € N the market value 
wg of its inventory: 


m 
Ws = Ss" Yidis. 
(=I 


An objection against a simple application of the principle (MP) 
could possibly be made if 


v(S) > Ss" ws holds for some coalition S C N. 
ses 


In this case, S could generate a market value that is strictly larger 
than the market value of its inventory. So the intrinsic economic 
value of the members of S is actually larger than the value of 
their inventory. This consideration leads to another worth assessment 
principle: 


(CA) Assign numbers ws to the members of N such that 


DEN = So ws and v(S)< Sy for all SCN. 
sEN ses 


An allocation w € R% according to principle (CA) is a so-called 
core allocation. Core allocations do not necessarily exist in a given 
cooperative game, however.° 

As it turns out, the principles (MP) and (CA) can be satisfied 
simultaneously if the production game (N,v) has a linear objective 


and linear restrictions. 


°Core allocations are studied more generally in Section 5. 
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Linear production games. Assume that the production game 
with characteristic function (50) is linear in the sense 


and admits an optimal production plan x* with market value 
v(N) = f(a*) =c' 2". 
x* is the solution of a linear program. So also an optimal solution y* 


exists for the dual linear program 


m 


m 
min, S > bj. yi s.t. Sai Pee [= Veates ks 
+ ij i=l 


where we have used the notation for the aggregated inventory of the 
members of a coalition: 


bp = d- dis for any SCN. 
ses 


The components y; of y* are the shadow prices of the materials Mj. 
According to principle (MP), let us allocate the individual worth 


m 
Ws = yf bis to any se N. 


i=1 
To see that w* satisfies also the principle (CA), observe first from 


linear programming duality: 


m 


k 
So wi = So oN yF = Soja = 0N). 
j=l 


sEN i=1 


The dual of any S-restricted production problem (50) differs only in 
the coefficients of the objective function. Expressed in terms of the 
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dual linear program, one has 


m 
8) = my Yo Us. BMs SS aijyi > cj os eee 
i=1 


yeRT 


Since y* is a feasible (although not necessarily optimal) dual solution 
for any S-restricted problem, one concludes: 


(8) < S08yf = Sout. 
i=1 


ses 


3.3. Network connection games 


Consider a set N = {p1,..., Dn} of users of some public utility® that 
are to be linked, either directly or indirectly (via other users), to some 
supply node po. Assume that the cost of establishing a link between 
p; with p; would be cj (euros, dollars or whatever). The associated 
cooperative game has the utility function 


c(S) = minimal cost of connecting just S to po. 


The relevant question is: 


e How much should a user p; € N be charged so that a network with 
the desired connection can be established? 


One possible cost distribution scheme is derived from a construction 
method for a connection of minimal total cost c(N): 


The greedy algorithm builds up a chain of coalitions 


C= 5) CSi-o SxCuccCS, EN 


according to the following iterative procedure: 


Like electricity or water, etc. 


146 &. Cooperative Games 


(Go) So = 9; 

(G;) If 5; has been constructed, choose p € N \ S; such that 
c(S; Up) is as small as possible and charge user p the 
marginal cost 


Opu(S) = 0S; U p) — c(S5). 


(Gr) Set S;41 = S; Up and continue until all users have been 
charged. 


The greedy algorithm makes sure that the user set N in total is 
charged the minimal possible connection cost: 


n n-1 
di le(Sj) — e(Sj-1)] = ($n) — (So) + SVfe(Sz) — e(Se)] 
j=l k=1 
= c(N) — c(0) 
=e). 


The greedy algorithm is efficient in the sense that the total cost c(V) 
is distributed. Nevertheless, the greedy cost allocation scheme may 
appear quite “unfair” from the point of view of individual users (see 
Ex. 8.7).! 


Ex. 8.7. Consider a user set N = {p1,p2} with connection cost 
coefficients co, = 100, cop = 101 and cyg = 1. The greedy algorithm 
constructs the coalition chain 


0= SoC S1 = {pi} C So = {p1- pe} = N 
and charges c(.S;) = 100 to user p;, while user p2’s marginal cost is 
c(S2) = c(S1) = 101 — 100 =1. 


So p; would be to bear more than 99% of the total cost c(N) = 101. 


"Game theorists disagree on “the best” network cost allocation scheme. 
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3.4. Voting games 


Assume there is a set N of n voters 7 of not necessarily equal voting 
power. Denote by w; the number of votes voter 7 can cast. Given 
a threshold w, the associated voting game® has the characteristic 


i if 7 wi > w 


v(S) = ies 
0 otherwise. 


function 


In the voting context, v(S) = 1 has the interpretation that the 
coalition S has the voting power to make a certain proposed measure 
pass. Notice that in the case v(.S') = 0, a voter i with marginal value 


O,v(S) = v(S Ut) — v(S) = 1 


has the power to swing the vote by joining S. The general question 
is of high political importance: 


e How can (or should) one assess the overall voting power of 


a voter i in a voting context? 


REMARK 8.3. A popular index for individual voting power is the 
BANZHAF power index (see Section 8 below). However, there are 
alternative evaluations that also have their merits. As in the case of 
network cost allocation, abstract mathematics cannot decide what 
the “best” method would be. 


4. Generalized coalitions and balanced games 


Let us assume that the TU-game (N,v) can be played by several 
coalitions S C N “simultaneously”, requiring an activity level ys > 0 
from every member i € S so that no player has to invest more than 
100% of its available activity resources in total. With this in mind, 


8 Also known as a threshold game. 
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we define a generalized coalition® to be a nonnegative vector 
y=(ys|SCN)ERY st. Sys<s1VieN 
Sdi 
and associate with it the utility value 
u(y) = (uly) = 55 v(S)ys. 
SCN 


Ex. 8.8. Assume that y = (ys|S C N) is a generalized coalition 
with binary components ys € {0,1}. Show that y is the incidence 
vector of a family of pairwise disjoint coalitions. 


Ex. 8.9 (Fuzzy coalitions). Let 7 = (7s|S C N) be a probability 
distribution on the family NV of all coalitions. Then one has 7g > 0 
for all S € N and 


Sons < Sims =1 for alli EN. 


S>i SEN 


So m represents a generalized coalition which generalizes the notion 
of a fuzzy coalition in the sense of Ex. 6.2. 


Denote by Y C RY the collection of all generalized coalitions y 
and note that VY is a nonempty, convex and compact set. The optimal 
utility value U is the optimal solution of a feasible linear program: 


v= maxv(y) = max v(S)ys  s.t. S "ys <1WieN. (51) 
+ SCN S31 


Taking y as the vector with components yy = 1 and yg = 0 if 
S 4 N, we see immediately: 


B > u(y”) = vo(N). 
The game (N,v) is called (positively) balanced if equality is achieved: 


w= O(N): 


° Also known as a packing. 
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The dual linear program associated with (51) has the same optimal 


value: 
T= min, x; s.t. Sa > v(S) VS CN. (52) 
TERY ICN iES 


Hence linear programming! duality yields: 


THEOREM 8.2 (BONDAREVA [5]). For any cooperative 
game (N,v), the two statements are equivalent: 


(1) (N,v) ts (positively) balanced. 


(2) For eachi € N there is a number x; > 0 such that 


uN) = SS. and See, >u(S) forall SCN. 
igN ieS 


Ex. 8.10. Let (N,v) be a balanced game. Show: 


u(N) = mareu(S): 


Ex. 8.11 (Linear production games). Show that a linear pro- 
duction game is positively balanced if and only if it admits an optimal 
production plan. 


Covers. The generalized coalition y = (ys|S C N) is said to cover 
the set N if equality 


S "ys =1 holds for all elements 7 € N, 
S31 


which means that each agent 2’s activity resource of unit value 1 is 
fully used under y. The covering value of (N,v) is the number 


v° = max {v(y) | y is a cover of (N,v)}. 


10 cf. Theorem 3.4. 
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As in the derivation of Theorem 8.2, we can characterize the covering 
value by linear programming duality and find 


Ex. 8.12. Prove formula (53). 


Clearly, one has v(N) < vo < 3. Calling the game (N,v) strongly 
balanced if it yields the equality 


v(N) =v, 


we therefore find: 


PROPOSITION 8.1. Every positively balanced game is strongly 


balanced. 


5. The core 
5.1. Stable sets 


Let us generally understand by a payoff vector of a cooperative game 
(N,v) an assignment x of individual values x; € R to the members 
i € N and hence a parameter vector « € RN. For any coalition 
S CN, we customarily use the notation 


iS) = Ss" on 
1ES 
for the total payoff to the members of S' under x. The payoff x is 
said to be feasible! if 


x(N) < v(N). 


"Here we assume again that v represents a gain. For a cost potential c, feasibility 
of an allocation « € RY means x(N) > c(N). 
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Solution concepts. One of the central issues of cooperative game 
theory is the question of defining (and then finding) appropriate 
payoffs to the players in a cooperative game (N,v). A concept for 
such payoffs is a solution concept. 

The classical solution concept suggested by VON NEUMANN and 
MORGENSTERN [834] consists in the identification of so-called stable 
sets of payoff vectors. 

To make this notion precise, say that x € R" dominates y € 
R% (relative to v) if there is a coalition S$ C N such that each of 
the members of S' is served better under x than under y without 
exceeding the value v(S), i.e., 


x; >y; holds forallic Sand 2(S) < v(S). 


In this context, a set S C RY is called stable if 


(1) All members of S are feasible. 
(2) Every feasible payoff z ¢ S is dominated by at least one 


x € S. (Hence S must be, in particular, nonempty.) 
(3) No « € S dominates any other y € S. 


Ex. 8.13 (VON NEUMANN—MORGENSTERN). Let N = {1,2,3} 
and consider the voting game [ = (N,v) with 


_ fi if|s|>2 
UO ic if |S] <2. 
Show that S = {(1/2,1/2,0), (1/2, 0, 1/2), (0,1/2,1/2)} is a stable 
set. 


As intuitively attractive as the solution concept of stable sets 
may appear, there are some practical drawbacks: 


e Not every cooperative game admits a stable set. 
e Even if stable sets exist, it may be difficult to identify one. 


In the sequel, we will discuss a related solution concept. Although 
it has similar practical drawbacks, its mathematical ramifications 


152 &. Cooperative Games 


are far-reaching and provide an important link to the mathematical 
theory of discrete optimization. 


5.2. The core 


Say that the payoff x € RY is coalition rational in the game (N,v) 
if each coalition is awarded at least its own value, 1.e., if 


x(S) >v(S)_ holds for all SCN. 


The core of a cooperative profit game (N,v) is the set of all feasible 
coalition rational payoff vectors: 


core(v) = {a € RN | 2(N) < v(N),2(S) > v(S) VS CN}. 


REMARK 8.4 (Efficiency). Note that every payoff vector x € 
core(v) is efficient in the sense 


x(N) = v(N). 


Ex. 8.14. Let x,y € core(v). Then x cannot dominate y because 
otherwise a coalition S' would exist with the property 


v(S) > x(S) > y(S) = v(S), 


which is a mathematical contradiction. 


PROPOSITION 8.2. Let S be an arbitrary stable set of the 
cooperative game (N,v). Then 


core(v) CS. 


Proof. Suppose to the contrary, that a vector y € core(v) \S exists. 
Since S is stable, it contains a payoff x € S that dominates y, i.e, 
there exists a coalition S C N so that 


v(S) < y(S) < #(S) < (5), 


which is impossible. 
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The core of a cost game (N,c) is defined analogously: 
core*(c) = {x € RN | a(N) > c(N), 2(S) < c(S) VS CN}. 


Every allocation x € core*(c) distributes the cost c(NV) among the 
players i € N so that no coalition S pays more than its proper cost 


c(S). 


Ex. 8.15. Show for the (zero-normalized) cooperative game (N,v) 
and its dual (NV, v*): 


core(v") = core” (v). 


Ex. 8.16. Give an example of a game (N,v) with core(v) = 0. 


PROPOSITION 8.3. Let (N,v) be an arbitrary TU-game. 
Then: 


(1) core(v) #0 <= =~ (N,v) is strongly balanced. 
(2) If v({i}) > 0 holds for alli € N in the game (N,v), then 


core(vl) #0 <= (N,v) ts positively balanced. 


Proof. Exercise left to the reader (cf. Theorem 8.2 and Proposition 
8.1). 
6. Core relaxations 


As Ex. 8.16 shows, the core is not a generally applicable concept 
for “fair” profit (or cost) distributions to the individual players in 
a cooperative game (N,v) because it may be empty. To deal with 
this drawback, various ways have been suggested to relax (and to 
strengthen) the idea of the core. 


6.1. The open core and least cores 


We associate with (N,v) the open core as the set 


core?(v) = {2 € R” | 2($) > v(S) VS CN}. 
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The set core°(v) is obviously never empty. Hence, every nonnegative 
parameter vector c € RY yields a feasible linear program 


min cla s.t. x € core®(v). (54) 


Notice that the objective function of (54) is bounded from below, 
because every x € core°(v) satisfies 


cg = Ss" rigs S- cu({i}). 
ieN icN 
Hence an optimal solution x* with an optimal value c* = c*(v) = 


cl x* exists. We call the set 


core(v,c) = {x € core®(v) | ca = c*} (55) 
the least c-core of (N,v). 
Ex. 8.17 (Classical least core). For c’ = (1,1,...,1) with com- 
ponents c; = 1, c* is precisely the covering value v° of (N,v) (cf. 
Section 4). The set 

core(u, u°) = {x € core®(v) | (NV) = v°} 

is the classical least core of (N,v).!? 
REMARK 8.5. The least core has a natural game-theoretic inter- 
pretation: Suppose that a coalition-rational payoff « € R% to the 
players incurs the cost c’a. Then core(v,c) is the collection of all 
cost-minimal coalition-rational payoffs. 
6.2. Nuclea 


The idea of the least core is a relaxation of the constraint 7(N) = 
v(N) while retaining the other core constraints x(S) > v(S). 

An alternative approach to a relaxation of the core concept 
consists in retaining the equality x(N)) = v(S) while possibly relaxing 
the other constraints. 


Further game-theoretic implications are studied in, e.g., FAIGLE and KERN [15]. 
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To make the idea precise, say that f € R™ is a relaxation vector if 
fo =0=fn and fs >0 for all coalitions Se N. 
f is feasible for v if there exists some scalar ¢ € R such that 
C(f,e) = core(u — ef) 49. 
Hence, if core(v) 4 0, every relaxation vector f is feasible (with « = 0, 


for example). 


LEMMA 8.1. Assume that f is a feasible relaxation with fs > 
0 for at least one S € N. Then there exists a scalar €9 € R 
such, that 


Cif.) #0 = €>6. 


Proof. Since f is feasible, the linear program 


os e st. 2(N) =v(N),efs +2(S) > v(S) VS EN 


is feasible. Moreover, the objective of the linear program is bounded 
from below since every feasible solution x satisfies 


vo(N) = 2(N) = 2(S)+2(N \S) 
2 v(S) + o(N \ S)— (fs + fr\s) 
and therefore 


Ss v(S) + u(N \ S) — v(N) 
= fst+fuvs 


So an optimal minimal value €9 with the desired property exists. 


If it exists, eg tightens the relaxation f to the best possible and yields 
the nonempty core relaxation 


Co = C(f,€0) = core(v — egf). 
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In the case Co 4 0, one may try to tighten the constraints further in 
the same way. To this end, we define the family 


Bo ={S EN | «(S) =a'(S) Vz,a" € Co}. 


Let ro = r(Bo) be the rank of the matrix of incidence vectors of 
the members of Bo. Since each incidence vector is n-dimensional and 
N € Bo holds, we have 


Ll<ro <n. 


LEMMA 8.2. If ro = n, then Co contains exactly one single 


vector xi. 


Proof. ro = n means that every S € N is a linear combination 
of the members of S € Bg. In particular the singleton coalitions 
{i} are such linear combinations. Since the values x(S) are uniquely 
determined for all S' € Bo, it follows that also the component values 
x; = x({i}) are uniquely determined for any x € Co. Hence there can 
be only one such z. 


In the case rg = n, we call the unique vector «/ € Co the f-nucleon 
of the game (N,v). 

If ro <n, let Ao be the family of coalitions S that are linearly 
independent of Bp. For every S € Ag, 


x(S) >v(S)—eofs holds for at least one x € Co. 
Hence there must exist a vector x € Cp such that 
x(S) > v(S)—egfs holds for all S € Apo. 


So these inequalities could possibly be tightened even further and 
one can try to find the best such €; < €9 as the optimal solution of 
the linear program 


mine st. £€Cpo,fgse+2(S) > v(S)VS € Apo. (56) 


€,x 
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LEMMA 8.3. €, exists if and only if fg > 0 holds for some 


SEAo. 


Proof. Completely analogous to the proof of Lemma 8.1 and left to 


the reader. 


If €, exists, we set 
C1 = Ci(f) = {a E RX | (a, 2) solves (56)} C Co(f) 
and 
By ={SEN | x(S) =2'(S) Vx € Ch}. 
Since S € B, holds for at least one S € Ag, we have 
n>r(Bi)=r1>rot+1. 


If r; =n, Ci(f) contains a singleton vector 2/, which we now term 
the f-nucleon of (N,v). 

In the case r; < n, the set A, of coalitions S that are not linearly 
dependent on 5, is non-empty. If fs > 0 holds for at least one S € Aj, 
we derive the existence of a minimal objective value 


€2 <€1 < € 
for the linear program 


mine st. £EC\,fge+2(S) > v(S) VS € Aj. (57) 


€,x 


in exactly the same way as before. And so on. 


After v < n iterations, we arrive at a situation where one of 


the two possibilities occurs: 


(1) CL(f) = {xf}. Then the vector vf is the f-nucleon of 


(N,v). 
(2) fs = 0 holds for all S € A,. Then the set C,(f) is the 
f-nucleon of (N,v). 
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Ex. 8.18. Show: core(v) #0 <= C,(f) C core(v). 


PROPOSITION 8.4. Let f be a feasible relaxation vector for 
the game (N,v) and F = {S € N | fs > O}. Then the f- 
nucleon set C,(f) has the property 


(CD eee 
(2) Ifr(F) =n, C.(f) consists of a singleton xf. 


6.3. Nucleolus and nucleon 


The nucleolus of the game (N,v) introduced by SCHMEIDLER [40] 
is the f'-nucleon relative to the relaxation vector f! with the unit 
parameters 


fg=1 for all coalitions S 4 0, N. 


By Proposition 8.4, it is clear that the nucleolus always exists and is 
a singleton.!% 
The nucleon of a game (N,v) with a nonnegative characteristic 


function is the f’-nucleon of the game relative to the relaxations 
x(S)>(1—e)v(S) forO<e<1 


i.e., the relaxation with the coefficients f’(.S) = v(S) for SAN. 
The choice € = 1 shows that the nucleon relaxation is feasible. 

The nucleon is a singleton vector x” if the (incidence vectors of the) 

coalitions S € N with value v(S) > 0 yield a system of full rank n. 


6.4. Excess minimization 


Let f be a relaxation vector for (N,v) and assume! 


fs >0 forall SAQ,N. 


3Related solution concepts are studied in MASCHLER et al. [28]. 

See FAIGLE et al. [18]. 

15To keep the discussion simple — the general case can be analyzed in the same 
spirit. 
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This assumption implies that f is a feasible relaxation and that (N,v) 
enjoys a unique f-nucleon z/. 

For any x € R% with x(N) = v(N), define the excess of a 
coalition S with fs > 0 as 


z(S}) — v(S) 
aks 


Since f is feasible, at least one such x exists so that e(x,S) > 0 holds 


E655) = 


for all excesses. Set 
F = {a € RN | 2(N) = v(N), e(2,S) > OVS ZO, N}. 


The parameter € 9 yields the minimal possible overall bound for the 
excess of a vector x € F and one has 


Co(f) = {a € F | e(z, S) > ep VS ZO, N}. 


One can now continue and determine the members of Co(f) with 
minimal excess and the family Ag of coalitions whose excess is not 
yet fixed. This yields the next parameter €, and the set 


Ci(f) = {x € Co(f) | e(z, $) > €1 VS € Ao}. 
And so on. This shows: 


e The f-nucleon «/ of (N,v) arises from a sequence of excess 
minimizations. 


In other words: 


The nucleon xf is the unique vector that distributes the value 


v(N) among the members of N so that each coalition ends 
with the minimal possible excess. 


There are further value assignment concepts in cooperative games 
that are of interest. Section 8 will provide examples of such concepts. 
For the moment, let us continue with the study of the core in its own 
right. 
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7. MONGE vectors and supermodularity 


In principle, it suffices to compute the nucleon xf of a cooperative 
game relative to a suitable relaxation vector f in order to have a core 
vector if the core is nonempty at all (cf. Ex. 8.18). The nucleon is 
typically difficult to compute, however. MONGE!® vectors are based 
on an alternative approach, which is computationally more direct. 
Games all of whose MONGE vectors lie in the core turn out to be 
particularly interesting. 

Throughout the discussion in this section, we consider a fixed 
zero-normalized cooperative game (N,v) with n players and collec- 
tion N of coalitions. 


7.1. The MONGE algorithm 


Given a parameter vector c € R% and an arrangement 7 = iji2... in 
of the elements of N, the MONGE algorithm constructs a primal 
MONGE vector 27 € R% and a dual MONGE vector y™ € R™ as 
follows: 


(M1) Set So = ) and $7 = Meee for k = ee ee >. 
(M2) Set xf = (SZ) — v(Sz_,) for k = 1,2,...,n. 


(M3) Set yg, = ci, and y§, = ci, —Ci,,, for @=1,2,...,n—1. 
Set yg = 0 otherwise. 


Notice that the construction of the primal MONGE vector x” proceeds 
like the greedy algorithm for the cost distribution in network 
connection games of Section 3 relative to the chain 


= S0-ES) Gus S aN: 


It is not hard to see!” that the MONGE vectors x” and y” satisfy the 
identity 


m™(c)= > _ gat = S° vo(S)y8. (58) 


iEN SEN 


‘6G. MoNGE (1746-1818). 
17 of Section 5 of the Appendix. 
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Different arrangements 7 and w of N, of course, may yield dif 
ferent MONGE sums m(c) and m*(c). Important is the following 
observation. 


LEMMA 8.4. Let m7 = iyi2...in and Ww = j1Jj2...jJn be two 
arrangements of N with non-increasing c-values: 


Cire 6s ee COG en Cpe Oh eaten Orie 


m*(c) 


Proof. Note that c, = ¢,,,, for example, implies yg, = 0. So we 
may assume that the components of c have pairwise different values. 
But then 7 = w holds, which renders the claim trivial. 


7.2. The MONGE extension 


Lemma 8.4 says that there is a well-defined real-valued function c > 
[u|(c) with the values 


lee) =m") for any aH 41 as eh eg ee Se 159) 


The function [v] : RY — R is called the MONGE extension of the 
characteristic function v: V > R. 

To justify the terminology “extension”, consider a coalition S' C 
N and its (0, 1)-incidence vector c(%) with the component ofS) =o) i 
the casei € S. 

An appropriate MONGE arrangement of the elements of N first 
lists all 1-components and then all 0-components: 


ieee ce ee eee S200 
Hence we have y§ = 1 and y= 0 for T # S and conclude 
[v](c)) = v(S) for all SCN, 


which means that @ and v agree on (the incidence vectors of) NV. 
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REMARK 8.6 (CHOQUET [8] and LovAsz [27]). 

Applying the idea and construction of the MONGE sums (58) to 
non-decreasing value arrangements f; < ... < fy, of nonnegative 
functions f : N — R4, one arrives at the CHOQUET integral 


[ f40= 3 telw(An) - ove), 
k=] 


where Ay = {k,k+1,...,n} and Ans = 0. 

The map f +» f fdv is the so-called LovAsz extension of the 
function v : N — R. Of course, mutatis mutandis, all structural 
properties carry over from MONGE to CHOQUET and LovASz. 


7.3. Linear programming aspects 
Generalizing the approach to the notion of balancedness of Theo- 
rem 8.2, let us consider the linear program 


min cz s.t. c(N) =v(N), 2(S) > v(S) if SAN. (60) 


xERN 


and its dual 


max v7 y s.t. ) ys <GViEN, ys > Oi SAN. (61) 
yER : 
Sdi 


for a given parameter vector c € R. Observe in the case 


that the dual MONGE vector y” relative to c is a dually feasible 
solution since yZ > O holds for all S # N. The feasible primal 
solutions, on the other hand, are exactly the members of core(v). 
Hence, if core(v) 4 9, both linear programs have optimal 
solutions. Linear programming duality then further shows 


ble) = min cle > vl y™=[v] (0). (62) 
xECOTe(v) 


THEOREM 8.3. 0 = [|v] holds for the game (N,v) if and only 


if all primal MONGE x” vectors lie in core(v). 
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Proof. Assume ¢, >... > G, and m7 = t,...%n. If e™ € core(v), 
then x” is a feasible solution for the linear program (60). Since the 
dual MONGE vector y” is feasible for (61), we find 


ch a™ > (ec) > [v](v) =cPa™ and hence i(c) = [v](c). 


Conversely, # = [v] means that the dual MONGE vector is guaranteed 
to yield an optimal solution for (61). So consider an arrangement w = 
jy, -.» jn of N and the parameter vector c € R“ with the components 


Cj, =n+1l—k fork=1,...,n. 


The dual vector y” has strictly positive components Us, =1>0on 


the sets oe It follows from the KKT-conditions for optimal solutions 
that an optimal solution x* € core(v) of the corresponding linear 
program (60) must satisfy the equalities 


a* (SY) = Ss" a v(S®) fork = lye 5M, 

ES, 
which means that x* is exactly the primal MONGE vector x” and, 
hence, that x” € core(v) holds. 


7.4. Concavity 


Let us call the characteristic function v : 2% > R concave if v arises 
from the restriction of a concave function to the (0,1)- incidence 
vector cS) of the coalitions S, i.e., if there is a concave function 
f :R’ > R such that 


v(S) = f(c)) holds for all S CN. 


Accordingly, the cooperative game (N,v) is concave if v is concave. 
We will not pursue an investigation of general concave cooperative 
games here but focus on a particularly important class of con- 
cave games which are closely tied to the MONGE algorithm via 
Theorem 8.3. 


PROPOSITION 8.5. Jf all MONGE vectors of the game (N,v) 


lie in core(v), then (N,v) is concave. 
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Proof. By Theorem 8.3, the hypothesis of the Proposition says 
&(c) = [v|(c) for allc e RY. 


Consequently, it suffices to demonstrate that 0 is a concave function. 
Clearly, Ac = No(c) holds for all scalars \ > 0, i.e., is positively 
homogeneous. 

Consider now arbitrary parameter vectors c,d € R™ and « € 
core(v) such that o(¢ +d) = (c+ d)x. Then 


b(e+d) =clx +d" x > i(c) + W(d), 


which exhibits 0 as concave. 


REMARK 8.7. The converse of Proposition 8.5 is not true: there 
are concave games whose core does not include all primal MONGE 
vectors. 


A word of terminological caution. The game-theoretic literature 
often applies the terminology “convex cooperative game” to games 
(N,v) having all primal MONGE vectors in core(v). In our terminol- 
ogy, however, such games are not convex but concave. 

To avoid terminological confusion, one may prefer to refer to such 
games as supermodular games (cf. Theorem 8.4 below). 


7.5. Supermodularity 


The central notion that connects the MONGE algorithm with con- 
cavity is the notion of supermodularity: 


THEOREM 8.4. For the cooperative game (N,v), the follow- 
ing statements are equivalent: 


(I) All MONGE vectors 2 € R lie in core(v). 


(II) v is supermodular, i.e., v satisfies the inequality 


V(SNT)+vu(SUT)>v(S)+v(L) forall S,TCN. 
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Proof. Assuming (I), arrange the elements of N to 7172...i, such 
that 


SoAT= {ig estet so = Lieu sigh lt ea nen 
By the definition of the MONGE algorithm, x” then satisfies 
gS) Hus il pe GS) =—os). 6 oT) ws Ur): 


Moreover, x"(T) > v(T) holds if x7 € core(v). So we deduce the 
supermodular inequality 


VSNT)+v0(SUT) =27(SNT)+a"(SUT) 
= 2"(S)+2"(T) 
> v(S) + v(T). 


Conversely, let 7 = 71,...,%, be an arrangement of N. We want to 
show: 


x” € core(v). 
Arguing by induction on n = |N|, we note x™() = v(0) and assume: 
x"($) >v(S) holds for all subsets S' of N’ = {i1,...,in—1}. 
Consequently, if i, € S, the supermodularity of v yields 
a(S) =27(SNN’) 4+ 27, 
= 2"(SNN’) +v(N) — v(N’) 
> o(S NN’) + v0(S UN’) — v(N’) 
> 008): 


Ex. 8.19. The preceding proof uses the fact that any vector x € RN 
satisfies the modular equality 


“(SAT)+a(SUT)=2($)+2(T) for all S,T CN. 


As SHAPLEY!® has observed, the concepts of core and stable sets 
coincide in supermodular games. 


181,.S. SHAPLEY (1923-2016). 
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THEOREM 8.5 (SHAPLEY [44]). The core of a supermodu- 


lar game (N,v) is a stable set. 


Proof. We already know that no core vector dominates any other 
core vector. Let y ¢ core(v) be an arbitrary feasible payoff vector. 
We claim that y is dominated by some core vector a. Consider the 
function 


g(S) = BO) on the family NV \ {0}. 


Since y ¢ core(v), the maximum value g* of g is strictly positive 
and attained at some coalition Z. Choose an order 7 = 71 ...%j...%n 
of the elements of N such that Z = {i1,...,i,} and let x7 be the 
corresponding primal MONGE vector. 

Define the payoff vector a € R% with components 


i ifj<k 


ai; = me 
vi, if 7 > k. 


I 
v 


Because «"(Z) = v(Z) = kg* + y(S), it is clear that a is feasible and 
dominates y relative to the coalition Z. It remains to show that a is 
a core vector. 

Keeping in mind that x” € core(v) holds (since v is supermodu- 
lar), we find for any coalition S: 


a(S) =a(SnZ)+a(Sn(N \ Z) 
=y(SNZ)+|(SNZ\g* +a7(SN(N \ Z)) 
> v(SNZ)+vu(SN(N \ Z)) 
> v(S). 


REMARK 8.8. Since the core of a supermodular game is stable and 
hence contains any other stable set, we know that core(v) is actually 
the unique stable set of the supermodular game (N,v). 
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7.6. Submodularity 


A characteristic function v is called submodular if the inequality 
v(SNT)+v0(SUT) < v(S)+v(L) holds for all $,T CN. 


Ex. 8.20. Show for the zero-normalized game (N,v) the equivalence 
of the statements: 


(1) v is supermodular. 
(2) v* is submodular. 
(3) w = —v is submodular. 


In view of the equality core(c*) = core*(c) (Ex. 8.15), we find 
that the MONGE algorithm also constructs vectors in the core*(c) 
of cooperative cost games (N,c) with submodular characteristic 
functions c. 


REMARK 8.9. Note the fine point of Theorem 8.4, which in the 
language of submodularity says: (NV,c) is a submodular cost game if 
and only if all MONGE vectors x” lie in core*(c). 

Network connection games are typically not submodular. Yet, 
the particular greedy cost distribution vector discussed in Section 3.3 
does lie in core*(c), as the ambitious reader is invited to demonstrate. 


REMARK 8.10. Because of the MONGE algorithm, sub- and super- 
modular functions play a prominent role in the field of discrete 
optimiziation.!9 In fact, many results of discrete optimization have a 
direct interpretation in the theory of cooperative games. Conversely, 
the model of cooperative games often provides conceptual insight 
into the structure of discrete optimization problems. 


REMARK 8.11 (Greedy algorithm). The Monge algorithm, 


applied to linear programs with core-type constraints, is also known 
as the greedy algorithm in discrete optimization. 


See, e.g., S. FUJISHIGE [20]. 
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8. Values 


While the marginal value 0;v(S) of player i’s decision to join 
respectively to leave the coalition S is intuitively clear, it is less clear 
how the overall strength of i in a game should be assessed. From a 
mathematical point of view, there are infinitely many possibilities to 
do this. 

In general, we understand by a value for the class of all TU-games 
(N,v) a vector-valued function 


&: RN 5 RY 


that associates with every characteristic function v a vector ®(v) € 
R%. Given ©, the coordinate value ©;(v) is the assessment of the 
strength of ¢ € N in the game (N,v) according to the evaluation 
concept ®. 


8.1. Linear values 


The value ® is said to be linear if ® is a linear operator, i.e., if one 
has for all games v,w and scalars A € R, the equality 


O(Av + w) = A®(v) + O(w). 


In other words: ® is linear if each component function ®; is a linear 
functional on the vector space RY. 

Recall from Ex. 8.3 that the unanimity games form a basis of 
R’’, which means that every game v can be uniquely expressed as 
a linear combination of unanimity games. Hence a linear value ® is 
completely determined by the values assigned to unanimity games. 
The same is true for any other basis of RY , of course. 

Indeed, if v1,...,v% € R% are (the characteristic functions 
of) arbitrary TU-games and )j,...,,% arbitrary real scalars, the 
linearity of ® yields 


OA V4 oa ae AVE) = A, ®(v1) reread Ap ®(up). 


We give two typical examples of linear values. 
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The value of SHAPLEY [43]. Consider the unanimity game or 
relative to the coalition T € NV where 


eS f£ ID 
in(S) = {7 ifS DT 


0 otherwise. 


In this case, it might appear reasonable to assess the strength of a 
player s € N\ T as null, i.e., with the value Cr) = 0, and the 
strength of each of the players t € JT in equal proportion as 


Zs 1 
OP" (67) = Va 


Extending ®%” to all games v by linearity in the sense 


one obtains a linear value v +> ®5"(v), the so-called SHAPLEY value. 


Ex. 8.21. Show that the players in (N,v) and in its zero- 
normalization (V,v**) are assigned the same SHAPLEY values. 


The power index of BANZHAF [2]. The BANZHAF power index 
6? appears at first sight to be quite similar to the SHAPLEY value, 
assessing the power value 


2 (57) =0 ifseN\T 
while treating all t € T as equals. Assuming T 4 0), the mathematical 
difference lies in the scaling factor: 


oP (57) = for allt € T. 


1 
giti-1 
As done with the SHAPLEY value, the BANZHAF power index is 
extended by linearity from unanimity games to all games (N,v) and 
thus gives rise to a linear value v 4 ®?(v). 

We will see in Section 8.2 that the difference between the values 
6°” and ®? can also be explained by two different probabilistic 
assumptions about the way coalitions are formed. 
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REMARK 8.12 (Efficiency). If |T| > 1, the SHAPLEY value dis- 
tributes the total amount 


> OF" (Gr) = 1 = dr(N) 

ieN 
to the members of N and is, therefore, said to be efficient. In contrast, 
we have 


me T x 
S07 (6r) = 7 <1=67(N) if |T| >2. 
ie N ani} 


So the BANZHAF power index in not efficient. 


REMARK 8.13. One can show that linear values for TU-games 
typically arise as so-called least square values, namely as values 
obtained from least square approximations with additive games 
under linear constraints.7? 


8.2. Random values 


The concept of a random value is based on the assumption that a 
player 7 € N joins a coalition S C N \ {i} with a certain probability 
7g as a new member. The expected marginal value of 1 € N thus is 


EXv)= S> dv(S)ns. 
SCNE{i} 


The function E™ : RY — R% with components E7(v) is the 
associated random value. 

Notice that marginal values are linear. Indeed, if u = Av+w, one 
has 


for alli € N and S CN. Therefore, the random value E” is linear 
as well: 


E™(\v +.w) = AE™(v) + E™(w). (63) 


20 cf. FAIGLE and GRABISCH [13]. 
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REMARK 8.14. The linearity relation (63) implicitly assumes that 
the probabilities 7s are independent of the particular characteristic 
function v. If mg depends on v, the linearity of E” is no longer 
guaranteed. 

The BOLTZMANN value (to be discussed in Section 9 below) 
is a random value that is not linear in the sense of (63) because 
the associated probability distribution depends on the characteristic 
function. 


8.2.1. The value of BANZHAF 


As an example, let us assume that a player i joins any of the 2"~! 
coalitions S C N \ {i} with equal likelihood, 7.e., with probability 


Consider the unanimity game vr = or and observe that O;vr(S') = 0 
holds if 7 ¢ T. On the other hand, if i € T, then one has 


OS) =1 Ss -T\ a CS. 
So the number of coalitions S with 0;vr(S) = 1 equals 
SON \ {i} | TC SU {| = 2. 


Hence we conclude 
gn—|T|-1 1 


B 
Ef (ur) = Ss" ivr (S)rg§ = gaat = 5 (64) 
SCN\{i} 


which means that the random value E™ is identical with the 
BANZHAF power index. The probabilistic approach yields the explicit 
formula 
1 
@F(v) = ET (vy) =— > S- (v(SUi)—v(8)) GEN). (65) 
SCN\{i} 
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8.2.2. Marginal vectors and the SHAPLEY value 
Let us imagine that the members of N build up the “grand coalition” 
N in a certain order 
CO =1112...Un 
and hence join in the sequence of coalitions 
O55. 6 Sy CS, Ceres, = N 
where S? = S?_, U {ix} for k = 1,...,n. Given the game (N,v), 7 
gives rise to the marginal vector?! 0°(v) € RN with components 
a, (0) = 0S.) oS a) (k= Leen): 


Notice that v ++ 07(v) is a linear value by itself. We can randomize 
this value by picking the order o from the set “iy of all orders of N 
according to a probability distribution 7. Then the expected marginal 
vector 


d*(v) = S > 8 (uv). 
o€UN 
represents, of course, also a linear value on RN. 
Ex. 8.22. Show that the value v ++ O7(v) is linear and efficient. 


(Hint: Recall the discussion of the greedy algorithm for network 
connection games.) 


THEOREM 8.6. The SHAPLEY value results as the expected 
marginal vector relative to the uniform probability distribution 
on in, where all orders are equally likely: 


55*(y) == S87 (v). 


*GEXN 


(Recall from combinatorics that there are n! = |Sn| ordered 
arrangements of N.) 


21The marginal vectors are precisely the primal MONGE vectors. 


8. Values 173 


Proof. Because of linearity, it suffices to prove the Proposition for 
unanimity games vp = op. For any order o = 7,...%, € Up and 
element i, € N \ T, we have 0;, (vr) = 0 and hence 


1 = 
7 ay OF (vr) =0 forallie N\T. 
o€UN 
On the other hand, the uniform distribution treats all members 7 € 


T equally and thus distributes the value vr(N) = 1 equally and 
efficiently among the members of T: 


1 . _ur(N)_ 1 
TT dy Fer) = iT) 7 Wr for allie T, 


o€UN 


which is exactly the concept of the SHAPLEY value. 


COROLLARY 8.2. The SHAPLEY value ©°” satisfies: 


(1) 6°9"(v) € core(v) if v is supermodular. 
(2) &9"(v) € core*(v) if v is submodular. 


Proof. A marginal vector is a primal MONGE vector relative to the 
zero vector c = O € RN. So all marginal vector lie in core(v) if v 
is supermodular. Since core(v) is a convex set and ®°”(v) a convex 
linear combination of marginal vectors, ®°"(v) € core(v) follows. 


We may interpret the SHAPLEY value within the framework of 
random values initially introduced. To this end, we assume that an 
order a € Uy is chosen with probability 1/n! and consider a coalition 
SCN \ {i}. We ask: 


e What is the probability mer that 7 would join S$? 


Letting sk — 1 = |S| be the size of 5, the number of sequences ao where 
i would be added to S is 


l{o |¢ =% and S?_, = S}| = (k—1)!(n—k)! 


This is so because: 
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(1) The first & — 1 elements must be chosen from S in any of the 
(k — 1)! possible orders. 
(2) The remaining n — k elements must be from N \ (SU {i}). 


So one concludes 


sh _ (k= Din—k)! _ ({S|(n = |S|—D! 


n! n! 


and obtains another explicit formula for the SHAPLEY VALUE: 


({S|!(n — | S|—1 
= FS ang = TS ams islea=isi=0 


SCN\ {i} SCN\{i} 


(66) 


Ex. 8.23. Consider a voting/threshold game (cf. Section 3.4) with 
four players of weights w, = 3, wa = 2,w3 = 2,w4 = 1. Compute the 
BANZHAF and the SHAPLEY values for each of the players for the 
threshold w = 4. 


9. Boltzmann values 


The probabilistic analysis of the previous section shows that the value 
assessment concepts of the BANZHAF power index and the SHAPLEY 
value, for example, implicitly assume that players just join — but 
never leave — an existing coalition in a cooperative game (N,v). 

In contrast, the model of the present section assumes an under- 
lying probability distribution 7 on the set 2% of all coalitions of N 
and assigns to player 7 € N its expected marginal value 


= » Oyv(S')1g5 


SCN 
Ex. 8.24. Let 7 be the uniform distribution on NV: 


1 
™ = 77 for all SEN. 
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In view of 
S > dv(S) = S- v(S) — o(S \ 4) + S$ (W(S) — o(S Ud) 
SCN ieS igS 
= SS) wfui)-o(7)) + SS @@)- (fv) 
TON\i TON\i 
=0, 
one has 


eo — Ss" O;0(S)xg = Wi iS Oxv(S 


SEN SEN 


So the expected marginal value of any particular player is zero, if all 
coalitions are equally likely. 


We further do allow 7 to depend on the particular characteristic 
function v under consideration. It follows that the functional v > 
E;(v,7) is not guaranteed to be linear. 

The idea of the BOLTZMANN value is based on the fact that one 
expects the characteristic value 


p= E(vu,7) = S> u(S)ag 
SEN 


if the players agree on a coalition S € N with probability 7g. So we 
may associate with yu its BOLTZMANN temperature T and define the 
aan: BOLTZMANN values 


a> a;v( (with Zp = Doe, e%)/T) (67) 
T SEN 


for the players i in the cooperative game (N,v) with expected 
characteristic value j. 


10. Coalition formation 


In the cooperative game (N,v) it may not be clear in advance which 
coalition S C N will form to play the game and produce the value 
v(S). The idea behind the theory coalition formation is the viewpoint 
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of a dynamic process in which players join and part in discrete time 
steps until a final coalition is likely to have emerged. 

If this coalition formation process has BOLTZMANN temperature 
T > 0, the METROPOLIS process suggests the model where, at the 
moment of a current coalition S C N, a randomly determined player 
1 € N considers to enact a possible transition 


i. PST SEES 
s+ saty= {509 ifid S 


by either becoming inactive or active. The transition is made with 
probability 


a4(S) = min{1, e%(9)/T}, 


In this case, the coalition formation process converges to the 
corresponding BOLTZMANN distribution on the family N of coalitions 
in the limit. 


Cost games. If (N,c) is a cost game, a player’s gain from the 
transition S + SA{z} is the negative marginal cost 


—0;c(S') = c(S') — c(SA{i}). 


So the transition S + SA{i} in the METROPOLIS process would be 
enacted with probability 
a’,(S) = min{1, e~ %9)/7 7}, 


v 


10.1. Individual greediness and public welfare 


Let us assume that N is a society whose common welfare is expressed 
by the potential v on the family N of all possible coalitions: If the 
members of N decide to join in a coalition S C N, then the value 
v(S) will be produced. 

If all members of N act purely greedily, an i € N has an 
incentive to change its decision with respect to the current coalition 
S depending on its marginal value 0;v(S) being positive or negative. 
This behavior, however, will not guarantee a high public welfare. 


10. Coalition formation 177 


The METROPOLIS process suggests that the public welfare can 
be steered if an incentive is provided such that 7 enacts a move S — 
SA{i} (i.e., changes its decision) with a non-zero probability 


al (S) = e%)/T (even) if d;v(S) < 0. 


If the control parameter T' > 0 is sufficiently small, the behavior 
of ani € N is “almost purely greedy” in the sense 


To0 => af(S)>30 if dv(S) <0. 


Moreover, a small temperature J > O in the coalition formation 
process allows us to expect a high public welfare. 


10.2. Equilibria in cooperative games 


In many cooperative games, the grand coalition offers an obvious 
equilibrium if the players’ utilities are assessed by their marginal 
values: 


LEMMA 8.5. Let (N,v) be a cooperative game. Then the two 
statements are equivalent: 


(1) N is a gain equilibrium with respect to the individual utility 
functions u;(S) = O;v(S). 
(2) v(N) > v(N \2) for allie N. 


In general, we may view (N,v) as an n-person matrix game with 
individual utilities 


uj(S) = 0;0(S) = v(SAz) — v(S). 


Hence we know from NASH’s Theorem 6.1 that the randomization of 
(N,v) admits an equilibrium. 


REMARK 8.15. The randomization of (N,v) means that each i € N 
selects a probability 0 < w; < 1 for the probability to become active. 
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The coalition S' is thus formed with probability 


w(S) =] [wu [ [a - w)). 


icS j¢S 
The expected value of v is thus 


E(v,w) = S© v(S)w(S). 


SCN 


The randomization of (V,v) essentially is a fuzzy game (see Ex. 6.2) 
with potential function 


B(v) = E(v,w) (w € [0,1]%). 


Observe, in contrast to the above: 


e The BOLTZMANN coalition formation model does not 


admit coalition equilibria at temperature T 4 0, unless 


v is constant, 


but implies high public welfare if the temperature is small. 

Many value concepts (like SHAPLEY and BANZHAF, for example), 
are based on marginal gains with respect to having joined a coalition 
as fundamental criteria for the individual utility assessment of a 
player. 

So let us consider the cooperative game (N,v) and take into 
account that the game will eventually split N into a group S$ C N 
and the complementary group S° = N \ S. Suppose a player i € N 
evaluates its utility relative to the partition (.S,S°) of N by 


v(S)-—v(S\i) ifie S 


rn oe fe —u(S°\i) ifie S* 
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Ex. 8.25. Assume that (N,v) is a supermodular game. Then one 
has for all players i 4 7, 


vi(N) = oN) — o(N \ 4) > oN \ 9) — O(N \ I) \ 4) = (NV 9) 
vi(N) = o(N) — o(N \ 4) 2 v({i}) — v0) = vi(N \ 4). 


Consequently, the grand coalition N represents a gain equilibrium 
relative to the utilities v;. 


Ex. 8.26. Assume that (N,c) is a zero-normalized submodular 
game and that the players 7 have the utilities 


c(S)—c(S\i) ifieS 
ei(S) = 
c(S°) —c(S°\2) ifie S®. 
Show: The grand coalition N is a cost equilibrium relative to the 
utilities c;. 
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Chapter 9 


Interaction Systems and Quantum Models 


This final chapter investigates game-theoretic systems making use of the 
algebra of complex numbers. Not only cooperation models are generalized, 
but also interaction of pairs on elements of a set X finds an appropriate 
setting. The states are naturally represented as hermitian matrices with 
complex coefficients. This representation allows one to carry out standard 
spectral analysis for interaction systems and provides a link to the 
corresponding mathematical model of quantum systems in physics. 


While the analysis could be extended to general HILBERT spaces, X is 
assumed to be finite to keep the discussion straigthforward.® 


It is historically perhaps surprising that JOHN VON NEUMANN, who laid 
out the mathematical foundations of quantum theory,’ did not build game 
theory on the same mathematics in his work with OSKAR MORGENSTERN. 


“See also FAIGLE and GRABISCH [12] 
’von NEUMANN [33] 


1. Algebraic preliminaries 


Since matrix algebra is the main tool in our analysis, we review 
some more fundamental notions from linear algebra (and Chapter 2). 
Further details and proofs can be found in any decent book on linear 
algebra.! 


‘e.g., NERING [31]. 
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Where X = {71,...,%m} and Y = {yi,...,yn} are two finite 
index sets, recall that R***’ denotes the real vector space of all 
matrices A with rows indexed by X, columns indexed by Y, and 
coefficients Az, € R. 

The transpose of A € R**~* is the matrix A? ¢ RY** with the 
coefficients Al, = A,,. The map A++ A? establishes an isomorphism 
between the vector spaces R*** and RY** . 

Viewing A € R**Y and B € R*Y** as mn-dimensional 
parameter vectors, we have the usual euclidian inner product as 


(AIB)= > AnyBey = (BTA), 
(x,y)EX XY 


where tr C denotes the trace of a matrix C. In the case (A|B) = 0, 
A and B are said to be orthogonal. The associated euclidian norm is 


|All = / (AIA*) = >) Acyl? 


(x,y)EX XY 


We think of a vector v € R* typically as a column vector. v! is 


the row vector with the same coordinates vi = vz. Be aware of the 
difference between the two matrix products: 


vv = Yo level? = [loll? 


rEx 
Up Ve, UtVag ose Up Vex, 
- UgoVa, VaoUng +++ Ux1Vam 
VU = 
| Unm Ur, VamUrg +° Al 


1.1. Symmetry decomposition 
Assuming identical index sets 
ROY SAG see } 


a matrix A € R*** is symmetric if Av = A. In the case A? = —A, 
the matrix A is skew-symmetric. With an arbitrary matrix 
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A € R***, we associate the matrices 
+_l T ers il T + 
A =g(At+A ) and A =g(A-A )=A-AT. 
Notice that A* is symmetric and A~ is skew-symmetric. The 
symmetry decomposition of A is the representation 


0s 


The matrix A allows exactly one decomposition into a symmetric 
and a skew-symmetric matrix (see Ex. 9.1). So the symmetry 
decomposition is unique. 


Bx Ode bet 4 b.o eR be such that A = 2-6. chow 
that the two statements are equivalent: 


(1) B is symmetric and C is skew-symmetric. 
(2) eee varie ba Ace 


Notice that symmetric and skew-symmetric matrices are necessarily 
pairwise orthogonal (see Ex. 9.2). 


Ex. 9.2. Let A be a symmetric and B a skew-symmetric matrix. 
Show: 


(AJB) =0 and ||A+ Bl? = Al? + IIB I?. 


2. Complex matrices 


In physics and engineering, complex numbers offer a convenient 
means to represent orthogonal structures. Applying this idea to the 
symmetry decomposition, one arrives at so-called hermitian matrices. 


Inner products. Recall that a complex number z € C is an 
expression of the form z = a+ ib where a and 0 are real numbers 
and iaspecial “new” number, the imaginary unit, with the property 


184 9. Interaction Systems and Quantum Models 


i? = —1. The squared absolute value of the complex number z = 
a+ib is 
|z|? = a? + b? = (a — ib)(a + ib) = Zz, 


with Z = a—ib being the conjugate of z. More generally, we define the 
hermitian product of two complex numbers u,v € C as the complex 
number 


(ulv) = Bu. (69) 


The (hermitian) inner product of two vectors u,v € C* with 
components uz, and vz is the complex number 


(ulv) = > (ueles). 


rEx 


The length (or norm) of a vector u = a+ ib € C* (with a,b € R*) 


18 
lel =f D2 (eteltte) = | SS fuel? =f D> laal? + [bel?. 
rEx rEX rEX 


Conjugates and adjoints. The conjugate of a vector v € C* is 
the vector 3 € C* with the conjugated components U,. The vector 
* 


v* =0" is the adjoint of v. With this notation, the inner product of 
the column vectors u,v € C* is 


(ulv) = Ss" U0 = OU; 
xEX 


where we think of the 1 x 1 matrix v*u just as a complex number. 


Accordingly, the adjoint of the matrix C € C**” is the matrix 


CAC Bere 


Ex. 9.3 (Trace). The inner product of the matrices U,V € C**® 
is 


(U|V) = tr (V*U). 
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The matrix C € C*** is selfadjoint if it equals its adjoint, i.e., if 
C=C*=C’. 


Ex. 9.4. Let v € C* be a column vector. Then vv* € C*** is a 
selfadjoint matrix of norm ||vv*|| = ||v||?. 


2.1. Spectral decomposition 


If (and only if) the matrix C ¢ C*** has real coefficients, 


C= 
holds and the notion “selfadjoint” boils down to “symmetric”. It is 
well-known that real symmetric matrices can be diagonalized. With 


the same technique, one can extend the diagonalization to general 
selfadjoint matrices: 


THEOREM 9.1 (Spectral Theorem). For a matrix C € 
C*** the two statements are equivalent: 


AW ies Scare Sat 
(2) C* admits a unitary basis U = {U, | « € X} of eigen- 
vectors Uz of C with real eigenvalues Ax. 


Unitary means for the basis U that the vectors U, have unit norm 
and are pairwise orthogonal in the sense 


_ fl ife=y 
Wale) = 45 ifaAy. 


The scalar A, is the eigenvalue of the eigenvector U, of C if 
CU, = r2Uz. 


It follows from Theorem 9.1 (see Ex. 9.5) that a selfadjoint matrix C 
admits a spectral? decomposition, i.e., a representation in the form 


?The spectrum of a matrix is, by definition, its set of eigenvalues. 
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C= SL, (70) 
rEX 


where the U, are pairwise orthogonal eigenvectors of C’ with 
eigenvalues A, € R. 


Ex. 9.5. Let U = {U, | x € X} be a unitary basis of C* together 
with aset A = {A, | « € X} aset of arbitrary complex scalars. Show: 


(1) The U, are eigenvectors with eigenvalues A, of the matrix 
C2 e Anu, 
rEX 


(2) C is selfadjoint if and only if all the A, are real numbers. 


The spectral decomposition shows: 


The selfadjoint matrices C in C*** are precisely the linear 
combinations of matrices of type 


Ca 
rex 


where the Uy are (column) vectors in C* and the rz are real 
numbers. 


Spectral unity decomposition. As an illustration, consider a 


C***  i.e., a matrix with pairwise orthogonal 


unitary matrix U € 
column vectors U, of norm ||U||, = 1, which means that the identity 


matrix J has the representation 
PaUW au. 


The eigenvalues of I have all value A, = 1. Relative to U, the matrix 
I has the spectral decomposition 


£25 UUs (71) 


rEx 
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For any vector v € C* with norm |jv|| = 1, we therefore find 


1 = (vlv) =v" Iv S- v*U,Uzo 


rex 
= 2 GSU) = YO Mole)? 
Lex rex 


It follows that the (squared) absolute values 
Pz =|(v|Uz)?_ (« € X) 


yield the components of a probability distribution p” on the set X. 
More generally, if the selfadjoint matrix C with eigenvalues p, has 
the form 


C= Ss" prU,Uz, 
LEX 
then we have for any v € C*, 
(u|Cv) = v*Cv = S> pel(v|Ue)? = >> pep’. (72) 
rEX rEX 


In other words: 


The inner product (v|\Cv) of the vectors v and Cv is the 


expected value of the eigenvalues py of C with respect to the 
probability distribution p” on X. 


Ex. 9.6 (Standard unity decomposition). The unit vectors 
e, € C*% yield the standard unity decomposition 


f= ) Cx€n- 
LEX 


Accordingly, a vector v € C* of length ||v|| = 1 with the components 
vz implies the standard probability distribution on X with the 
components 


Pz = |(vlex)|? = lve)? 
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2.2. Hermitian representation 


Coming back to real matrices in the context of symmetry decompo- 
sitions, let us associate with a real matrix A € R*** the complex 
matrix 


ASAT AGAR, 


A is a hermitian® matrix. The hermitian map At> A establishes an 


isomorphism between the vector space R*** and the vector space 


Hx ={A|AeR***} 


with the set R as field of scalars.4 The import in our context is the 
fundamental observation that the selfadjoint matrices are precisely 
the hermitian matrices: 


LEMMA 9.1. Let C € C*** be an arbitrary complex matrix. 
Then 


Cel —] C=C 


Proof. Assume C = A+iB with A, B € R*™* and hence 
C* = AT —iBt 


So C = C* means symmetry A = A? and skew-symmetry B = —BY’. 
Consequently, one has A = A and B = iB, which yields 


C=A+iB=A+BeHy. 


The converse is seen as easily. 


3C. HERMITE (1822-1901). 
“Hx is not a complex vector space: The product zC of a hermitian matrix C with 
a complex scalar z is not necessarily hermitian. 
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The remarkable property of the hermitian representation is: 


e While a real matrix A € R*** does not necessarily 
admit a spectral decomposition with real eigenvalues, its 


hermitian representation A is always guaranteed to have 


one. 


Ex. 9.7 (HILBERT space). Let A,B € R*** be arbitrary real 
matrices. Show: 


(A|B) = (AB), 


i.e., inner products (and hence norms) are preserved under the 
hermitian representation. This means that R*** and Hy are not 
only isomorphic as real vector spaces but also as (real) HILBERT 
spaces. 


3. Interaction systems 


Let us assume that elements x,y € X can interact with a certain 
interaction strength, measured by a real number az,. We denote this 
interaction symbolically as dzyézy. Graphically, one may equally well 
think of a weighted (directed) edge in an interaction graph with X 
as its set of nodes: 


ee Any 
Gea 2 QO ® 


An interaction instance is a weighted superposition of interactions: 


e= S dant ay: 


vjyEex 


We record the interaction instance ¢ in the interaction matrix A € 


R*** with the interaction coefficients Azy = Gzy. The interaction 


is symmetric if AT = A and skew-symmetric if AT = —A. 
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R*x*x 


Conversely, each matrix A € corresponds to some inter- 


action instance 
E= y AgyExy- 
xjyEex 


R*** as the interaction space relative to the set 


X. Moreover, the symmetry decomposition 


So we may think of 


A=At+A™ 
shows: 
Every interaction instance € on X is the superposition 


=e" 2" 


of a symmetric interaction instance et and a skew-symmetric 


interaction instance e«~. Moreover, et and €¢~ are uniquely 
determined. 


Binary interaction. In the binary case |X| = 2, interaction 
matrices are (2 x 2)-matrices. Consider, for example, the matrices 


1 0 0 1 
i; 4 and tm=|° il 


I is symmetric and Im is skew-symmetric. Superposition of these 
two matrices with real scalars a,b € R yields the interaction matrix 


z= at +bIm= | ° ). 
—b a 


which is the matrix representation of the complex number z = a+ib. 

Note that the binary interaction space is 4-dimensional, however. 
So the complex numbers only describe a 2-dimensional subclass of 
binary interactions. 
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3.1. Interaction states 


The norm of an interaction state ¢ with interaction matrix A is the 
norm of the associated interaction matrix: 


llell = ||All- 
So |le|| 4 0 means that at least two members s,t € X interact with 
strength A,, ~ 0 and that the numbers 


|Aayl* 


Pay = Al? ((a, y) Ex x X) 


yield a probability distribution on the set of all possibly interacting 
pairs and offer a probabilistic perspective on e: 


e A pair (x,y) of members of X is interacting nontrivially 


with probability pry. 


Clearly, scaling ¢ to Ae with a scalar 4 4 0, would result in the same 
probability distribution on X x X. From the probabilistic point of 
view, it therefore suffices to consider interaction instances ¢ with 
norm |le|| = 1. 


We thus define: 


The interaction system on X is the system 3(X) with the set 
of states 


Jx = {e | € is an interaction instance of X of norm |le|| = 1} 


In terms of the matrix representation of states, we have 


3x 49 Sx ={AeR*** | ||A] =1}. 


3.2. Interaction potentials 


A potential F' : X x X — R defines a matrix with coefficients F,,, = 
F(a,y) and thus a scalar-valued linear functional 


Aw (F|A) = aS FayAgy 
r,yEX 
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on the vector space R***. Conversely, every linear functional f on 


R**~ is of the form 


f(A) = S FryAry = (F|A) 
rjyEex 


with uniquely determined coefficients F;,, € R. So potentials and 
linear functionals correspond to each other. 

On the other hand, the potential F defines a linear operator 
A+ FeA on the space R***, where the matrix Fe A is the 
HADAMARD product of F and A with the coefficients 


(Fe A)sy = FryAcy for all x,y € X. 


With this understanding, one has 


(FIA) = SO (Fe Ay 


rjyEex 


Moreover, one computes 


xjyex vjyEex 


Summarizing, one finds: 


If A € Sx (i.e., if A represents an interaction state ¢ € 3x), 
the parameters Da = |A,,|? define a probability distribution 
on X x X. The expected value of the potential F' in this state 
€ is 


= © Faypi, = (AIF e A). 
x YyEX 


3.3. Interaction in cooperative games 


The interaction model offers a considerably wider context for the 
analysis of cooperation. 

For an illustration, consider a cooperative TU-game T = (N,v) 
with collection N of coalitions. v is a potential on VV but not on the 
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set N x N of possibly pairwise interacting coalitions. However, there 
is a straightforward extension of v to N x N: 


NON od 


Relative to a state o € Jyyxy with interaction matrix A, the expected 
value of v is 


o(0) = > v(S)|Ass/?. 


SEN 


In the special case of a state og where the coalition S interacts with 
itself with certainty (and hence no proper interaction among other 
coalitions takes place), we have 


v(os) = v(S) 


which is exactly the potential value of the coalition S in the classical 
interpretation of [. 


Generalized cooperative games. A more comprehensive model 
for the study of cooperation among players would be structures of 
the type 


=, (N, N, v), 


where v is a potential on WV x NV (rather than just NV’). This point of 
view suggests to study game-theoretic cooperation within the context 
of interaction. 


3.4. Interaction in infinite sets 


Much of the current interaction analysis remains valid for infinite 
sets with some modifications. 

For example, we admit as descriptions of interaction states only 
those matrices A € R*** with the property 


(H1) supp(A) = {(a,y) € X x X | Ary # 0} is finite or countably 
infinite. 


(H2) ||A? = SO Aryl? = 


vjyEex 
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If the conditions (H1) and (H2) are met, we factually represent 
interaction states in HILBERT spaces. To keep things simple, however, 
we retain the finiteness property of the agent set X in the current text 
and refer the interested reader to the literature® for further details. 


4. Quantum systems 


Without going into the physics of quantum mechanics, let us 
quickly sketch the basic mathematical model and then look at the 
relationship with the interaction model. In this context, we think of 
an observable as a mechanism a that can be applied to a system 6, 


P-L] 


with the interpretation: 


e If G is in the state o, then a is expected to produce a 


measurement result a(c). 


4.1. The quantum model 


There are two views on a quantum system Qyx relative to a set 
X. They are dual to each other (reversing the roles of states and 
observables) but mathematically equivalent. 


The SCHRODINGER picture. In the so-called SCHRODINGER® 
picture, the states of Qy are presented as the elements of the set 


Wx = {ve C* | lv = 1} 


of complex vectors of norm 1. An observable a corresponds to a 
(selfadjoint) matrix A € Hx and produces the real number 


a(v) = (v|Av) = v* A*u = U* Av 


°e.g., HALMOS [23] or WEIDMANN [46]. 
°&. SCHRODINGER (1887-1961). 
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when Qy is in the state v € W. Recall from the discussion of the 
spectral decomposition in Section 2.1 that a(v) is the expected value 
of the eigenvalues p, of A relative to the probabilities 


pe” = |(vlUx)/? (w € X), 


where the vectors Uz € W constitute a vector space basis of 
corresponding unitary eigenvectors of A. 
An interpretation of the probabilities p4” could be as follows: 


Oy is a stochastic system that shows the element x € X with 
probability pe if it is observed under A in the state v: 


. ~[A]—= 


The elements of X are weighted with the eigenvalues of A. The 
measurement value is the expected weight of the x produced 
under A. 


Ex. 9.8. The identity matrix IJ € C*** yields the standard 
probabilities’ 


pe’ = lve (« € X). 


The HEISENBERG picture. In the HEISENBERG® picture of Qy, 
the selfadjoint matrices A € Hy take over the role of states while 
the vectors v € W induce measurement results. The HEISENBERG 
picture is dual? to the SCHRODINGER picture. In both pictures, the 
expected values 


(v|Av) (vu € Wx,A € Hx) 
are thought to be the numbers resulting from measurements on Qy. 
"of. Ex. 9.6. 


SW. HEISENBERG (1901-1976). 
°In the sense of Section 2.1. 
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The HEISENBERG picture sees an element x € X according to 
the scheme 


4 > [J-= 


with probability pa. 


Densities and wave functions. The difference of the two pictures 
lies in the interpretation of the probability distribution p4” on the 
index set X relative to A € Hy and v € Wx. 

In the HEISENBERG picture, p“” is imagined to be implied by a 
possibly varying A relative to a fixed state vector v. Therefore, the 
elements A € Hx are also known as density matrices. 

In the SCHRODINGER picture, the matrix A is considered to be 
fixed while the state vector v = v(t) may vary in time t. v(t) is called 
a wave function. 


4.2. Evolutions of quantum systems 


A quantum evolution 
® = 0(M,v, A) 


in (discrete) time t depends on a matrix-valued function t > M;, a 
state vector v € W, and a density A € Hx. The evolution ® produces 
real observation values 


vp =u"(MtAM,)v (t =0,1,2,...). (74) 


Notice that the matrices A, = MAM; are selfadjoint. So the 
evolution ® can be seen as an evolution of density matrices, which is 
in accordance with the HEISENBERG picture. 

If v(t) = Myv € W holds for all t, the evolution © can also 
be interpreted in the SCHRODINGER picture as an evolution of state 
vectors: 


Yt = (M,v)* A(Myv) (t = 0, 1, 2; oe Se (75) 
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REMARK 9.1. The standard model of quantum mechanics assumes 
that evolutions satisfy the condition M;v € W at any time t, so that 
the HEISENBERG and the SCHRODINGER pictures are equivalent. 


Markov coalition formation. Let N be the collection of coalitions 
of the set N. The classical view on coalition formation sees the prob- 
ability distributions p on N as the possible states of the formation 
process and the process itself as a MARKOv!® chain. For example, 
the METROPOLIS process! is an example of a MARKOV chain. 

To formalize this model, let % = ‘B(N) be the set of all 
probability distributions on V. A MARKOV operator is a linear map 


ww: No 3 RY such that yp € B holds for all p € . 


u defines for every initial state p© € $8 a so-called MARKOV chain 
of probability distributions 


M(p) = {u'(p) | t = 0,1,...}. 


Define now P, € RY** as the diagonal matrix with p® = pt(p) 
as its diagonal coefficient vector. P; is a real symmetric matrix and 
therefore a density in particular. 

Any v € W gives rise to a quantum evolution with observed 
values 


Tao Pe (eS Oe ye o): 


For example, if eg € R™ is the unit vector that corresponds to the 
coalition S € N, one has 


(S) 


nm) = e& Reg = (Piss =p 


with the usual interpretation: 


104A. MARKOV (1856-1922). 
"cf. Chapter 2. 
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e If the coalition formation proceeds according to the 
Markov chain M(p), then an inspection at time t 


es ae te coalition S to be active with probability 
S t 
TM]  =Pg- 


4.3. The quantum perspective on interaction 
Recalling the vector space isomorphism via the hermitian represen- 
tation 


AeR***-, A=AtIA € Hx, 


we may think of interaction states as manifestations of SCHRODINGER 
states of the quantum system Qxy x, 


Sx ={AeER*™ | A] = 1} & Wxxx = {Ae Hx | |All = 15, 


or as normed representatives of HEISENBERG densities relative to the 
quantum system Qy. 


Principal components. An interaction instance A on X has a 
hermitian spectral decomposition 


A= SU Up =o Anas 


rex rEX 


where the matrices A, = U,Uz are the principal components of 


A. The corresponding interaction instances A, are the principal 
components of A: 


A= Ss" ArAr- 


rEx 


Principal components V of interaction instances arise from 
SCHRODINGER states v = a+ib € Wx with a,b € R* in the following 
way. Setting 


V = vv* = (a+ ib)(a — ib)? = aa? — bb" + i(ba? — ab’), 


4. Quantum systems 199 


one has V+ = aa! + bb” and V~ = ba? — ab’, and thus 
VSVIRV- =(aa> 0") (ba? ab"). 
The principal interaction instance V has hence the underlying 


structure: 


(0) Each x € X has a pair (az, b,) of weights az, by, € R. 
(1) The symmetric interaction between two arbitrary elements 
x,y € X is 


Vi = GzQy + deby. 


ay 
(2) The skew-symmetric interaction between two arbitrary 
elements x,y € X is 


i = by — Azby. 


4.4. The quantum perspective on cooperation 


Let N be a (finite) set of players and family N of coalitions. From 
the quantum point view, a (SCHRODINGER) state of N is a complex 
vector v € Wy, which implies the probability distribution p” with 
probabilities 


pi =lvi? (ie N) 


on N. In the terminology of fuzzy cooperation, p” describes a fuzzy 
coalition: 


Conversely, if w € RN is a non-zero fuzzy coalition with component 
probabilities 0 < w; < 1, the vector 


Vw = (fui |t € N) 


may be normalized to a SCHRODINGER state 


Jw 2 
v= —— st. we=|lVull- lui) for allie N. 
I Vw| 
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In the same way, a vector Wy describes a SCHRODINGER state of 
interaction among the coalitions of N. It is particularly instructive 
to look at the interactions V of principal component type. 

As we have seen in Section 4.3 above, V arises as follows: 


(0) The interaction V on N is implied by two cooperative 
games [, = (N,a) and T, = (N,0). 
(1) Two coalitions S,T € N interact symmetrically via 


Vep = a(S)a(T) + b(S)0(T). 
(2) Two coalitions S,T € N interact skew-symmetrically via 


Vey = 0(S)a(T) — a($)0(T). 


5. Quantum games 


A large part of the mathematical analysis of game-theoretic systems 
follows the guideline: 


e Represent the system in a mathematical structure, analyze 


the representation mathematically and re-interpret the 
result in the original game-theoretic setting. 


When one chooses a representation of the system in the same space 
as the ones usually employed for the representation of a quantum 
system, one automatically arrives at a “quantum game”, i.e., at a 
quantum-theoretic interpretation of a game-theoretic environment. 
So we understand by a quantum game any game on a system G 
whose states are represented as quantum states and leave it to the 
reader to review game theory in this more comprehensive context. 


6. Final remarks 


Why should one pass to complex numbers and the hermitian space 
Hx rather than the euclidian space R*** 
isomorphic real Hilbert spaces? 


if both spaces are 
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The advantage lies in the algebraic structure of the field C of 
complex numbers, which yields the spectral decomposition (70), for 
example. It would be not impossible, but somewhat “unnatural” to 
translate this structural insight back into the environment R*** 
without appeal to complex algebra. 

Another advantage becomes apparent when one studies evolu- 
tions of systems over time. In the classical situation of real vector 
spaces, MARKOV chains are important models for system evolutions. 
It turns out that this model generalizes considerably when one passes 
to the context of HILBERT spaces.!” 

The game-theoretic ramifications of this approach are to a large 
extent unexplored at this point. 


2 PalGLe and GIERz [11]. 
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Appendix 


1. Basic facts from real analysis 


(More details can be found in the standard literature.!) The euclidian 
norm (or geometric length) of a vector « € R” with components 2, 


is 
lvl] = 4/2? +...4+- 22. 


The ball with center x and radius r is the set 
B(x) = {y ER" | |lz —y|| <r}. 


A subset S C R” is closed if each convergent sequence of elements 
zy, € S has its the limit point x is also in S: 


tpoau = «es. 


The set S is open if its complement R” \ S is closed. The following 
statements are equivalent: 


(O’) S is open. 
(O”) For each x € S' there is some r > 0 such that B,(x) CS. 


The set S is bounded if S C B,(0) holds for some r > 0. S is said 
to be compact if S' is bounded and closed. 


te.g., RUDIN [39]. 
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LEMMA A.2 (HEINE-BOREL). S C R” is compact if and only if 


(HB) every family O of open sets O C R” such that every x € S$ 
lies in at least one O € O, admits a finite number of sets 
O1,...,0¢ € O with the covering property 


e SCO,UOQU...UOz. 


It is important to note that compactness is preserved under 
forming direct products: 


e If X CR” and Y C R™ are compact sets, then X x Y C R™*™ 
is compact. 


Continuity. A function f : S — R” is continuous if for all 
convergent sequences of elements xz; € 5S, the sequence of function 
values f(x) converges to the value of the limit: 


tpae => f(t) > f(a). 
The following statements are equivalent: 


(C’) f : S > R™ is continuous. 
(C”) For each open set O C R™, there exists an open set O’ C R” 
such that 


f-\(O) ={weS| f(z) EO}=O'CS, 


i.e., the inverse image f~!(O) is open relative to S. 
(C”’) For each closed set C C R™, there exists a closed set C’ C R” 
such that 


fC) ={zeES| f@)eC}=C'CS, 
i.e., the inverse image f~!(C) is closed relative to S. 


LEMMA A.3 (Extreme values). Jf the real-valued function f : 
S > R is continuous on the nonempty compact set S C R”, then 
there exist elements x,,x* € S such that 


f(xx) < f(a) < f(x") holds for alla € S. 
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Differentiability. The function f : S —> R is differentiable on the 
open set S C R” if for each x € S there is a (row) vector V f(x) such 
that for every d € R” of unit length ||d|| = 1, one has 


tim Le +#@) ~ Fle) _ ,, Vil@dd 
t 


t ER). 
t0 t t0 (fe ) 


V f(x) is the gradient of f. Its components are the partial derivatives 
of f: 

Vi (e) = (Of(a)/Ar1,...,9f(@)/Ian). 
Nota BEng. All differentiable functions are continuous — but not 


all continuous functions are differentiable. 


2. Convexity 


A linear combination of elements 21,...,2m is an expression of the 
form 


z=Aya,+...+AmeZm,; 


where 1,...,Am are scalars (real or complex numbers). The linear 
combination z with scalars A; is affine if 


Apt... +Am = 1. 


An affine combination is a convex combination if all scalars A; are 
nonnegative real numbers. The vector A = (Aj,...,Am) of the m 
scalars A; of a convex combination is a probability distribution on the 
index set 


Convex sets. The set S C R” is convex if it contains with every 
x,y € S also the connecting line segment: 


[x,y] ={e+rAy—a2)|O0<A<1}CS. 
It is easy to verify: 


e The intersection of convex sets yields a convex set. 
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e The direct product S$ = X x Y C R”*™ of convex sets _X C R” 
and Y C R”™ is a convex set. 


Ex. A.9 (Probability distributions). For X = {1,...,n}, the 
set 


P(x =A ei ete) ERs) Oy he, a 
of all probability distributions on X is convex. Because the function 
Fl Bisevh slg) Ar a3 ey 
is continuous, the set 
$0) SS fies eA eR” | fGen) = 


is closed. The collection R of nonnegative vectors is closed in R”. 
Since the intersection of closed sets is closed, one deduces that 


P(X) = f'(LOR} C B,(0) 
is closed and bounded and thus compact. 


Convex functions. A function f : S — R is conver (up) on the 
convex set S if for all x,y € S and for all scalars 0 < A < 1, 


flc+Ay—«)) = f(x) + AF) — Fle). 


This definition is equivalent to the requirement that one has for any 
finitely many elements 21,...,%_ € S$ and probability distributions 


(Aisne aa) 
fOw1 +... + Amem) > Arf (@1) +--+ Amf (Em). 


The function f is concave (or convex down) if g = —f is convex (up). 
A differentiable function f : S — R on the open set S C R” is 
convex (up) if and only if 


f(y) > f(a) +VF(x)(y—x) holds for allz,yeS. (76) 
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Assume, for example, that Vf(x)(y — x) > 0 it true for all y € S, 
then one has 


f(x) = min f(y). 


yes 


On the other hand, if Vf(x)(y — x) < 0 is true for some y € S, one 
can move from x a bit into the direction of y and find an element 
x’ with f(x’) < f(x). Hence one has a criterion for minimizers of f 
on S: 


LEMMA A.4. If f is a differentiable convex function on the convex 
set S, then for any x € S, the statements are equivalent: 


(1) f(@) = min f(y). 
(2) Vf(«)\(y— 2) >0 for ally ES. 

If strict inequality holds in (76) for all y 4 x, f is said to be 
stricly convex. 


In the case n = 1 (1.e., S C R), a simple criterion applies to twice 
differentiable functions: 


fisconvex = f"(x)>0 forallxzeS. 


For example, the logarithm function f(x) = Inz is seen to be strictly 
concave on the open interval S = (0,00) because of 


f'(z)=-1/2? <0 forallzeS. 


3. Polyhedra and linear inequalities 


A polyhedron is the solution set of a finite system of linear equalities 
and inequalities. More precisely, if A C R™”*” is a matrix and b € R™ 
a parameter vector, then the (possibly empty) set 


P(A,b) = {a € R” | Az < b} 


is a polyhedron. Since the function x +> Az is linear (and hence 
continuous), one immediately checks that P(A, b) is a closed convex 
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subset of R”. Often, nonnegative solutions are of interest and one 
considers the associated polyhedron 


P,(A,b) = {@ € R4 | Ax < b} = {a € R” | Ax < b,-Ia < 0}, 
with the identity matrix I € R"*”. 


Ex. A.10. The set P(X) of all probability distributions on the finite 
set X is a polyhedron. (cf. Ex. A.9.) 


LEMMA A.5 (FARKAS’). If P(A,b) £40, then for any c € R” and 
z ER, the following statements are equivalent: 


(1) c’x < z holds for all x € P(A,b). 
(2) There exists some y > 0 such that y1 A =c" and y’b < z. 


Lemma A.5 is a direct consequence of the algorithm of FOURIER,? 
which generalizes the Gaussian elimination method from systems of 
linear equalities to general linear inequalities.* 

The formulation of Lemma A.5 is one version of several equi- 
valent characterizations of the solvability of finite linear (in)equality 
systems, known under the comprehensive label FARKAS Lemma. 
A nonnegative version of the FARKAS Lemma is: 


LEMMA A.6 (FARKAS+). If P;,(A,b) 4 0, then for any c € R” 
and z ER, the following statements are equivalent: 


(1) ca < z holds for all x € P,(A,b). 
(2) There exists some y > 0 such that y2A>c? and y"b < z. 


Ex. A.11. Show that Lemma A.5 and Lemma A.6 are equivalent. 
Hint: Every x € R” is a difference x = xt — x~ of two nonnegative 
vectors xt,x~. So 


Ar<b 4 Axt—Ax <b. 


?Gy. Farkas (1847-1930). 
3J. FouRIER (1768-1830). 
4See, e.g., Section 2.4 in FAIGLE et al. [17]. 
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4. BROUWER’s fixed-point theorem 


A fixed-point of a map f : X > X is a point x € X such that 
f(x) = a. It is usually difficult to find a fixed-point (or even to decide 
whether a fixed-point exists). Well-known sufficient conditions were 
formulated by BROUWER’: 


THEOREM A.2 (BROUWER). Let X C R” be a convex, compact 
and non-empty set and f : X > X a continuous function. Then f 
has a fixed-point. 


Proof. See, e.g., the enyclopedic text of GRANAS and DUGUNDJI 
[22]. 


For game-theoretic applications, the following implication is of 
interest. 


CORROLLARY A.1. Let X C R” be a convex, compact and 
nonempty set andG: X x X +R a continuous map that is concave 
in the second variable, i.e., 


(C) for every x € X, the map y+> G(a2,y) is concave. 
Then there exists a point x* € X such that for ally © X, one has 


G(a", a") > G(a",y). 


Proof. One derives a contradiction from the supposition that the 
Corollary is false. Indeed, if there is no x* with the claimed property, 
then each x € X lies in at least one of the sets 


Oly) ={@ € X | Gia, a) < Gla,y)} (YE X). 


Since G is continuous, the sets O(y) are open. Hence, as X is 
compact, already finitely many cover all of X, say 


X C O(y1) U O(y2) U... U O(yn)- 


°L.E.J. BROUWER (1881-1966). 
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For all x € X, define the parameters 
d(x) = max{0, G(z, yz) —G(xz,xz)} (€=1,...,h). 
x lies in at least one of the sets O(y¢). Therefore, we have 
d(x) = dy(a) + do(x) +...+dp(x) > 0. 
Consider now the function 
j 
xr v(x) = So d0(x)ys (with Ap = de(x)/d(a)). 
é=1 


Since G is continuous, also the functions x ++ d¢(x) are continuous. 
Therefore, y : X — X is continuous. By BROUWER’s Theorem A.2, 
yy has a fixed-point 


8 
* 
I 
aS) 
— 
8 
* 
4 
I 
Nia 
~- 
oS 
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8 
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oS 


l=1 


Since G(x, y) is concave in y and «* is an affine linear combination 
of the ye, we have 


If the Corollary were false, one would have 
NeG(x™, ye) > Ae(z*)G(a*, x*) 

for each summand and, in at least one case, even a strict inequality 
Ae(x")G(x*, ye) > AeG(x*, x*), 


which would produce the contradictory statement 


It follows that the Corollary must be correct. 
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5. The MONGE algorithm 
The MONGE algorithm with respect to coefficient vectors c,v € R” 
has two versions. 
The primal MONGE algorithm constructs a vector x(v) with the 
components 


fv H=wp and! eg) = oe Se RSH 28 se 


The dual MONGE algorithm constructs a vector y(c) with the 
components 


yn(C) =n and y(c)=ce—cor (C=1,...,n-D) 


Notice: 


The important property to observe is as follows: 


LEMMA A.7. The MONGE vectors x(v) and y(c) yield the equality 


= s Chtp(v) = Ss ueye(c) = v7 y(c). 
k=1 é=1 


Proof. Writing x = x(v) and y = y(c), notice for all 1 <k,@ <n, 
Qy+tgt+...+2g=ve and YR+YRpit---+Yn = Ck; 


and hence 


n £L 
Ss" Ckhlk = 3 = Yere = aD LeYe = 3 veVe. 
k=1 1 k=1 


k=1 l=k 
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6. Entropy and BOLTZMANN distributions 
6.1. BOLTZMANN distributions 


The partition function Z for a given vector v = (v1,...,Un) of real 
numbers v; takes the (strictly positive) values 


Z(t) = ST ett (t € R). 
j=l 


The associated BOLTZMANN probability distribution b(t) has the 
components 


bj (t) = e’i" /Z(t) > 0 
and yields the expected value function 


Hit) = 2 opbj() = 
j=l 


The variance of v is its expected quadratic deviation from u(t): 


One has o?(t) 4 0 unless all v; are equal to a constant K (and hence 
u(t) = K for all t). Because p/(t) = o?(t) > 0, one concludes that 
u(t) is strictly increasing in t unless p(t) is constant. 

Arrange the components of v such that vj < vg <... < un. Then 


t 
lim ——— = lim e(%-*)* = 0 unless Uj = Un, 
which implies b;(t) — 0 if vj < wn. It follows that the limit 
distribution b(0co) is the uniform distribution on the maximizers of v. 


Similarly, one has 


(¢ 
oe) = lim e%-")'=Q unless Uj = UI 
t+—o00 by (t) t+—oo 
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and concludes that the limit distribution b(—co) is the uniform 
distribution on the minimizers of v. 


THEOREM A.3. For every value vy < € < Un, there is a unique 
parameter t such that 


€ = u(t) =D) vybj(t). 
j=l 


Proof. The expected value function p(t) is strictly monotone and 
continuous on R and satisfies 


lim p(A) =u < w(t) <u, = lim pw). 
A009 


A—>+00 


So, for every prescribed value € between the extrema vj and vy, there 


must exist precisely one t with p(t) = €. 


6.2. Entropy 


The real function h(x) = xlnz is defined for all nonnegative real 
numbers® and has the strictly increasing derivative 


hi (x) =1+Inz. 
So h is strictly convex and satisfies the inequality 
h(y) — h(x) > h'(x)(y—-) for all nonnegative y # x. 


h is extended to nonnegative real vectors x = (21,...,%n) via 


Day = NGin<s. ta) = >, Bln = So h(a;) 
j=1 j=l 


The strict convexity of h becomes the inequality 
h(y) — h(w) > Vh(a)(y— «), 
with the gradient 
Vh(x) = (h'(21),...,h'(an)) = (1 +1nazy,...,1+1nzp). 


®With the understanding In0 = —oo and 0-In0=0. 
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In the case 21 +... +2, = 1, the nonnegative vector x is a 
probability distribution on the set {1,...,n} and has” the entropy 


= Dam (L/a;) = -Leimay = —h(x1,...,%n)- 


We want to show that BOLTZMANN probability distributions are 


precisely the ones with maximal entropy relative to given expected 
values. 


THEOREM A.4. Let v = (v1,...,Un) be a vector of real numbers 
and b the BOLTZMANN distribution on {1,...,n} with components 
= Leet Ze 
I~ Za = ree | 
with respect to some t. Let p = (pj,.--,;Pn) be a probability 


distribution with the same expected value 


a UjPj = ~ 0b; 
Then one has either p = b or H(p) < an 


Proof. For d = p— b, we have }),d; =), pj — 01,0; =1-1=0, 
and therefore 


Vh(b)d = 3 d;(1 + Inb;) 


i 
= = or ut + (1-InZ(t)) So dy =t S° vyd; 
j=l j=l j=l 
a) So vip; — S > vj; =t(— pp) =0 
j=l j=l 


In the case p ¥ b, the strict convexity of h thus yields 
h(p) — h(b) > Vh(b)(p — b) =O and hence H(p) < H(b). 


"By definition! 
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LEMMA A.8 (Divergence). Let a1,...,@n,Pi,---;Pn be arbitrary 
nonnegative numbers. Then 


n n n n 
as don = Sopa <> pnp. 
i=1 i=1 i=1 i=1 


Equality is attained exactly when a; = p; holds for alli =1,...,n. 


Proof. We may assume p; # 0 for all 7 and make use of the well- 
known fact (which follows easily from the concavity of the logarithm 
function): 


Inz<a-1 and Intx=x-1sxr=1. 


Then we observe 
n n n n 
Loin < Soi (= = i) = So ai ~Sopi <0 
i=l ‘i=l Pi i=l i=l 
and therefore 
n n n 
S— pina, = Soi In pj = do pins S10; 
i=1 i=l i=1 


Equality can only hold if In(a;/p;) = (a;/p;) — 1, and hence a; = p; 
is true for all 7. 


7. MARKOV chains 


A (discrete) MARKOV chain® on a finite set X is a (possibly infinite) 
random walk on the graph G = G(X) whose edges (x, y) are labeled 
with probabilities 0 < pz, < 1 such that 


Se for alae X. 
yEex 


8For more details, see, e.g., KEMENY and SNELL [25]. 
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The walk starts in some node s € X and then iterates subsequent 
transitions x + y with probabilities pzy: 


a a a a a a 


Let P = [pry] be the matrix of transition probabilities and P” = 


[pS] the n-fold matrix product of P. Then the random walk has 
reached the node y after n iterations with probability 


pe) (ye X). 


In other words: The zo-row of P” is a probability distribution p™ 
on X. 

The MaArRKOV chain is connected if every node in G can be 
reached from any other node with non-zero probability in a finite 
number of transitional steps. This means: 


e There exists some natural number m such that pom) > 0 holds 
for alla,y EX. 


LEMMA A.9. If the MARKOV chain is connected and pyz > 0 holds 
for at least one x € X, then the MARKOV chain converges in the 
following sense: 


(1) The limit matrix P? = lim P” exists. 
Noo 


(2) P® has identical row vectors p™). 
(3) As a row vector, p'°) is the unique solution of the (in)equality 
system 


pP=p with py >0 and Se pel 
cEex 


A useful sufficient condition for the computation of a limit 
distribution p'©) of a MARKOV chain is given in the next example. 


Ex. A.12. Let P = [pry] € R*** be a Markov transition 
probability matrix and p € R* a probability distribution on X 
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such that 
PePey = PyPry is true for all x,y € X. 


Then P’p = p holds. Indeed, one computes for each x € X: 


SS) PyPey = SS" PePry = Px SS yx = py 1 = py. 


yEex yEex yEx 
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